FR Doc 06-3101

[Federal Register: April 12, 2006 (Volume 71, Number 70)]
[Notices]
[Page 18925-19052]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr12ap06-143]

[[Page 18925]]

-----------------------------------------------------------------------

Part III

Election Assistance Commission

-----------------------------------------------------------------------

Technical Guidelines Development Committee (TGDC); Initial Report:
Voluntary Voting System Guidelines Version I; Notice

[[Page 18926]]

-----------------------------------------------------------------------

ELECTION ASSISTANCE COMMISSION

Technical Guidelines Development Committee (TGDC); Initial
Report: Voluntary Voting System Guidelines Version I

agency: United States Election Assistance Commission.

action: Notice; publication of TGDC recommendations for voluntary
voting system guidelines.

-----------------------------------------------------------------------

summary: The Help America Vote Act of 2002 (HAVA) Section 221(f)
directs the Technical Guidelines Development Committee (TGDC) to
publish its recommendations to the Executive Director of the U.S.
Election Assistance Commission (EAC) at the time EAC adopts voluntary
voting system guidelines. In 2004, the EAC formed the TGDC to create an
initial set of recommendations for guidelines as directed by HAVA. The
Director of the National Institute of Standards and Technology (NIST)
chairs the TGDC and NIST staff provides technical support for the
TGDC's work. This committee of fifteen experts began their work in July
2004 and submitted their recommendations, which are published here.
These recommendations were used by the EAC in producing the EAC's
proposed 2005 Voluntary Voting System Guidelines which were published
for public comment in June 2005, 70 FR 37378 (June 29, 2005). Following
revision of its proposed guidelines to reflect the comments received,
the EAC adopted the final 2005 Voluntary Voting System Guidelines on
December 13, 2005. This final document is being concurrently published
as required by HAVA.

for further information contact: Brian Hancock (Election Research
Specialist) Washington, DC, (202) 566-3100, Fax: (202) 566-3127.

Thomas R. Wilkey,
Executive Director, U.S. Election Assistance Commission.
BILLING CODE 6820-KF-P

[[Page 18927]]

[GRAPHIC] [TIFF OMITTED] TN12AP06.011

[[Page 18928]]

Voluntary Voting System Guidelines Version I

Initial Report

May 9, 2005

Product of the Technical Guidelines Development Committee With technical
Assistance From the National Institute of Standards and Technology
------------------------------------------------------------------------

-------------------------------------------------------------------------
Overview:
Volume One, Performance Standards:
Section One: Introduction
Section Two: Functional Capabilities
Section Three: Hardware
Section Four: Software
Section Five: Telecommunications
Section Six: Security
Section Seven: Quality Assurance
Section Eight: Configuration Management
Section Nine: Overview of Qualification Testing
Appendix A: Glossary
Appendix B: Applicable Documents
Appendix C: Best Practices
Appendix D: Independent Dual Verification
Volume Two, Testing Standards:
Section 1: Introduction
Section 2: Technical Data Package
Section 3: Functionality Testing
Section 4: Hardware Testing
Section 5: Software Testing:
Section 6: Systems Integration Testing
Section 7: Configuration Management and Quality Assurance
Appendix A: Qualification Test Plan
Appendix B: Qualification Test Report
Appendix C: Qualification Test Design Criteria
------------------------------------------------------------------------

Voluntary Voting System Guidelines--Overview

This section provides an overview of the Voluntary Voting System
Guidelines (VVSG), Version 1. The VVSG was created in response to the
Help America Vote Act (HAVA) of 2002 and is based on the initial set of
recommendations of the Technical Guidelines Development Committee
(TGDC) mandated by HAVA. The VVSG Version 1 augments the Voting Systems
Standard (VSS) of 2002 (VSS-2002), which was promulgated by the Federal
Election Commission (FEC). This overview serves as an explanation of
how the VVSG Version 1 differs from the VSS-2002 and provides a basis
for further improvements. In addition, it provides a high level
overview of the major sections of the two volumes that make up VVSG
Version 1.

Document Structure

This document presents the voluntary voting system guidelines as a
single document consisting of two volumes: Volume I, the performance
provisions of the guidelines and Volume II, the testing specification.
Sections of this document augment the VSS-2002, by either replacing
VSS-2002 sections or adding new sections. New material is indicated by
distinct header information on each page. The header information is in
a gray shaded box and includes the words ``NEW MATERIAL''. The footer
information also includes the words ``NEW MATERIAL''. Additionally,
line numbers have been added to these pages.
In the new sections that contain requirements or informative
characteristics, each requirement or characteristic is numbered
according to a hierarchical scheme in which higher-level requirements
(such as ``provide accessibility for blind voters'') are supported by
lower level requirements (``provide an audio-tactile interface'').
These sections are: Sections 2.2.7, 6.0.1, 6.0.2, 6.0.3, 6.0.4, and
Appendix D. Additionally, each requirement or characteristic indicates
to whom it applies (i.e., responsible entity) as well as which stage of
the voting process (i.e., pre-voting, voting, post-voting) is affected.
There are three responsible entities: voting system vendor (V), testing
authority (T), and repository (R). To aid the reader, a colored box
with the first letter of the responsible entity, i.e., V, T, or R
accompanies the name of the entity, as follows:
[GRAPHIC] [TIFF OMITTED] TN12AP06.000

The three stages of the voting process are indicated by a
presenting a box with all three stages and using a strikeout font to
indicate the stages that are not applicable, as follows:
[GRAPHIC] [TIFF OMITTED] TN12AP06.001

Indicates the pre-voting stage is the only stage that applies.
[GRAPHIC] [TIFF OMITTED] TN12AP06.002

Indicates all three stages apply.

Background

The Help America Vote Act (HAVA) established the Technical
Guidelines Development Committee to assist the Election Assistance
Commission (EAC) with the development of voluntary voting system
guidelines. HAVA directs the National Institute of Standards and
Technology (NIST) to chair the TGDC and to provide technical support to
the TGDC in the development of these guidelines. The TGDC's initial set
of recommendations for these guidelines were presented to the Election
Assistance Commission in May 2005, in accordance with HAVA's nine-month
deadline.
VVSG Version 1 is intended to assist State election officials in
preparing for the 2006 election. This document augments the VSS-2002 to
address the critical areas of accessibility, usability and computer
security. In addition, the VVSG includes an improved glossary to
promote common understanding, a conformance clause, and an updated
Appendix on error rates.
It is important to note that the VVSG Version 1 is an interim set
of guidelines. The EAC is working with both the TGDC and NIST to create
a redesigned VVSG (called VVSG Version 2) that will address a large
range of issues including rewriting the requirements, if necessary, to
make them more precise and testable and address key human factors and
computer security issues. These new requirements will affect the basic
design of voting systems to such a degree that these types of changes
cannot reasonably be made and tested in time for the 2006 election
cycle.

Brief History of Voting Systems Standards and Guidelines

In 1975, the National Bureau of Standards (now the National
Institute of Standards and Technology) and the Office of the Federal
Elections (the Office of Election Administration's predecessor at the
General Accounting Office) produced a joint report, Effective Use of
Computing Technology in Vote Tallying. This report concluded that a
basic cause of computer-related election problems was the lack of
appropriate technical skills at the state and local level to develop or
implement sophisticated Standards against which voting system hardware
and software could be tested. A subsequent Congressionally-authorized
study produced by the FEC and the National Bureau of Standards detailed
the need for a federal agency to develop national performance Standards
that could be used as a tool by state and local election officials in
the testing, certification, and procurement of computer-based voting
systems.
In 1984, Congress appropriated funds for the FEC to develop
voluntary national Standards for computer-based voting systems. The FEC
formally approved the Performance and Test Standards for Punchcard,
Marksense and Direct Recording Electronic Voting Systems in January
1990. This document is generally referred to as the Voting Systems
Standards, or 1990 VSS.
The national testing effort was developed and overseen by the
National

[[Page 18929]]

Association of State Election Director's Voting Systems Board, which is
composed of election officials and independent technical advisors.
NASED's testing program was initiated in 1994 and more than 30 voting
systems or components of voting systems have gone through the (NASED's)
testing and qualification process. In addition, many systems have
subsequently been certified at the state level using the Standards in
conjunction with functional and technical requirements developed by
state and local policymakers to address the specific needs of their
jurisdictions.
As the qualification process matured and qualified systems were
used in the field, the Voting Systems Board, in consultation with the
testing labs, was able to identify certain testing issues that needed
to be resolved. Moreover, rapid advancements in information and
personal computer technologies introduced new voting system development
and implementation scenarios not contemplated by the 1990 Standards.
In 1997, NASED briefed the FEC on the necessity for continued FEC
involvement, citing the importance of keeping the Standards current in
its reflection of modern and emerging technologies employed by voting
system vendors. Following a Requirements Analysis released in 1999, the
Commission authorized the Office of Election Administration to revise
the Standards to reflect contemporary needs of the elections community.
This resulted in the 2002 Voting Systems Standards.
In 2002, Congress passed HAVA, which created a new process for
improving voluntary voting system guidelines. A new federal entity was
created, the Election Assistance Commission, to oversee the process.
The EAC established the Technical Guidelines Development Committee in
accordance with the requirements of section 221 of HAVA pursuant to the
Federal Advisory Committee Act, 5 U.S.C. App. 2. The TGDC's objectives
and duties were to act in the public interest to assist the EAC in the
development of the voluntary voting system guidelines. The membership,
as defined by HAVA, includes:
The Director of the National Institute of Standards and
Technology (NIST) who shall serve as its chair,
Members of the Standards Board,
Members of the Board of Advisors,
Members of the Architectural and Transportation Barrier,
and Compliance Board (Access Board),
A representative of the American National Standards
Institute,
A representative of the IEEE,
Two representatives of the NASED selected by such
Association who are not members of the Standards Board or Board of
Advisors, and who are not of the same political party, and
Other individuals with technical and scientific expertise
relating to voting systems and voting equipment.
The TGDC first met in August, 2004 and delivered the Voluntary
Voting System Guidelines in May, 2005. This initial set of
recommendations augments the VSS-2002 by including security measures
for auditability, wireless communications and software distribution and
setup, and improvements to the accessibility and usability design
sections of the VSS-2002. The TGDC also recommended that the VSS-2002
be replaced with a far-reaching guideline that would address in-depth
security, performance-based guidelines for usability testing, and an
overhaul of the standards and test methods to meet today's more
rigorous needs for electronic voting systems.

Issues Addressed by the VVSG Version 1

The VVSG Version 1 adds or significantly changes eight technical
topics of the VSS-2002. In addition, there are three organizational
changes in the new sections. All other material remains the same.

Conformance Clause

The VSS-2002 did not include a conformance clause. One has been
written and inserted as Section 1.7. The previous material in Section
1.7, the Outline, has been moved to 1.8.
Conformance is defined as the fulfillment by a product, process, or
service of requirements as specified in a standard or specification.
Conformance testing is the determination of whether an implementation
(i.e., product, process, or service) faithfully satisfies the
requirements and thus, conforms.
The conformance clause of a standard specification is a high-level
description of what is required of implementers and developers. It, in
turn, refers to other parts of the standard. The conformance clause may
specify minimal requirements for certain functions and minimal
requirements for implementation-dependent values. It may also specify
the permissibility of extensions, options, and alternative approaches
and how they are to be handled.

Human Factors

In the VSS-2002 Volume 1 Section 2.2.7 addressed Accessibility and
Section 3.4.9 addressed Human Engineering--Controls and Displays. The
VSS-2002 also contained Appendix C on Usability. The VVSG Version 1
replaces all of these items with a new Section 2.2.7 that addresses
Human Factors including accessibility, usability, and limited English
proficiency. This new sections incorporates the two NASED Technical
Guides (Guide 1 and Guide 2). Future versions of the
VVSG will contain performance-based requirements.

Security Overview and Appendix D

A new security section was added as Section 6.0. It contains four
parts: an Overview and three topic areas. The overview was added to
explain the VVSG approach to security. Future versions of the VVSG will
require independent dual verification. There are many ways known today
to achieve independent dual verification and more ways may be
developed. Current methods include dual process systems, witness
systems, cryptographic-based systems, optical scan systems, and paper
audit trails. A new Appendix D expands on this overview with an in-
depth discussion of independent dual verification systems. Independent
dual verification is a new area in voting systems and it is expected to
evolve significantly in VVSG Version 2. The Security Overview is an
informative (non-normative) section of the VVSG Version 1. Requirements
for voter verified paper audit trail systems, which are a type of
independent dual verification system, are specified in a separate
section. Version 2 of the VVSG will have complete requirements for at
least three additional methods.

Voter Verified Paper Audit Trails

The VSS-2002 contained no requirements for voter verified paper
audit trails. The VVSG Version 1 is providing requirements for voter
verified paper audit trails (VVPAT) so that States that choose to
implement VVPAT or States that are considering implementation can
utilize these requirements to help ensure the effective operation of
these systems. The EAC, TGDC, and NIST are taking no position with
respect to the implementation of VVPAT systems and are neither
requiring nor endorsing voter verified paper audit trails. Methods
other than VVPAT can provide ways to achieve independent dual
verification. These other methods are described in the Security
Overview.

Wireless Technology

The TGDC concluded that the use of wireless technology introduces
risk and

[[Page 18930]]

should be approached with caution. Therefore, the VVSG Version 1
includes a new section on wireless that augments the general
telecommunications requirements in Volume 1, Section 5. in Section 5.
The VVSG Version 1 requires that wireless transmissions be encrypted to
protect against a variety of security problems.

Software Distribution and Setup Validation

The VSS-2002 contains many requirements to help voting officials
validate the software and the setup of voting system software and
hardware. Subsequent to the publication of the VSS-2002, the EAC
invited all voting software vendors to submit their software to a
national software repository maintained by NIST. This section of the
VVSG Version 1 builds on the VSS-2002 to include use of this repository
and other validation mechanisms.

Glossary

Error Rates

Volume II, Appendix C addresses error rates. This appendix contains
revised procedures to test that systems meet the indicated error rates.
These apply to errors introduced by the system, defined as a ballot
position error rate, and not by a voter's action. Further research on
human interface and usability issues is needed to enable the
development of Standards for error rates that account for human error.
There were concerns about the VSS-2002 Appendix regarding the
numbers listed in the probability ratio sequential test (PRST) of the
Mean Time Before Failure (MTBF) that (1) the numbers do not correspond
to the numbers for the same table in the 1990 VSS, even though the
stated assumptions do not change, and (2) the numbers from neither the
1990 nor the 2002 tables correspond to numbers that would result from
standard PRST formulas listed in standard references such as the
military handbook MIL-HDBK-781A. To address these concerns, the revised
Appendix has replaced the numbers in the table with those that would
indicated by the truncated PRST design from MIL-HDBK-781A with the
corresponding parameters and made it more clear in the text that a
truncated design was chosen. Using standard theoretical formulas leads
to somewhat different numbers, but the revised Appendix C uses numbers
from the MIL-HDBK-781A because they may be considered more standard and
produce a less drastic change. Also, in the 1990 VSS, there was an
appendix devoted to the definition and use of ``partial failures.''
This appendix was eliminated from the VSS-2002. The new version
eliminated the paragraph and diagram in Appendix C that used partial
failures.
The new version also includes statements reminding users to be
cognizant of the assumptions involved in tests that use time-based
exponential failure times and constant failure rates. Given the
concerns that have been stated about appropriate testing times, note
that the given table is appropriate only for the stated parameters, and
that officials should assess the appropriateness of whatever parameters
are used in testing.

Best Practices for Voting Officials

The VSS-2002 contained requirements for voting systems and for
testing entities. However, requirements for human factors, wireless
communications, VVPAT, software distribution and setup validation
depend not only on voting systems providing specific capabilities but
on voting officials developing and carrying out appropriate procedures.
Consequently, the VVSG Version 1 contains Best Practices for voting
officials. The new sections in VVSG Version 1 define each requirement
as pertaining to voting systems, vendor repository, or test
authorities, or voting officials. The requirements for voting officials
are collected in Appendix C of Volume 1. (Appendix C had previously
been Usability.)

Voting Process

The VSS-2002 defined three major stages of voting: pre-voting,
voting, and post-voting. The stage for each requirement is marked in
the new sections. The VVSG Version 2 will have a more detailed voting
process model and will allow for finer granularity.

Summary of Content of Volume I

Volume I contains performance standards for electronic components
of voting systems. In addition to containing a glossary (Appendix A),
applicable references (Appendix B), Best Practices (Appendix C) and
Security Overview (Appendix D). Volume I is divided into nine sections:
Section 1--Introduction: This section provides an introduction to
the Standards, addressing the following topics:
Objectives and usage of the Standards,
Development history for initial Standards,
Update of the Standards,
Accessibility for individuals with disabilities,
Definitions of key terms,
Application of the Standards and test specifications,
Conformance clause, and
Outline of contents.
Section 2--Functional Capabilities: This section contains Standards
detailing the functional capabilities required of a voting system. This
section sets out precisely what it is that a voting system is required
to do. This section also sets forth the minimum actions a voting system
must be able to perform to be eligible for qualification. For
organizational purposes, functional capabilities are categorized by the
phase of election activity in which they are required:
Overall Capabilities: These functional capabilities apply
throughout the election process. They include security, accuracy,
integrity, system auditability, election management system, vote
tabulation, ballot counters, telecommunications, and data retention.
Pre-voting Capabilities: These functional capabilities are
used to prepare the voting system for voting. They include ballot
preparation, the preparation of election-specific software (including
firmware), the production of ballots or ballot pages, the installation
of ballots and ballot counting software (including firmware), and
system and equipment tests.
Voting Capabilities: These functional capabilities include
all operations conducted at the polling place by voters and officials
including the generation of status messages.
Post-voting Capabilities: These functional capabilities
apply after all votes have been cast. They include closing the polling
place; obtaining reports by voting machine, polling place, and
precinct; obtaining consolidated reports; and obtaining reports of
audit trails.
Maintenance, Transportation and Storage Capabilities:
These capabilities are necessary to maintain, transport, and store
voting system equipment.
For each functional capability, common standards are specified. In
recognition of the diversity of voting

[[Page 18931]]

systems, some of the standards have additional requirements that apply
only if the system incorporates certain functions (for example, voting
systems employing telecommunications to transmit voting data) or
configurations (for example, a central count component). Where system-
specific standards are appropriate, common standards are followed by
standards applicable to specific technologies (i.e., paper-based or
DRE) or intended use (i.e., central or precinct count).
Section 3--Hardware Standards: This section describes the
performance requirements, physical characteristics, and design,
construction, and maintenance characteristics of the hardware and
related components of a voting system. This section focuses on a broad
range of devices used in the design and manufacture of voting systems,
such as:
For paper ballots: Printers, cards, boxes, transfer boxes,
and readers,
For electronic systems: Ballot displays, ballot recorders,
precinct vote control units,
For voting devices: Punching and marking devices and
electronic recording devices,
Voting booths and enclosures,
Equipment used to prepare ballots, program elections,
consolidate and report votes, and perform other elections management
activities,
Fixed servers and removable electronic data storage media,
and
Printers.
The Standards specify the minimum values for the relevant
attributes of hardware, such as:
Accuracy,
Reliability,
Stability under normal environmental operating conditions
and when equipment is in storage and transit,
Power requirements and ability to respond to interruptions
of power supply,
Susceptibility to interference from static electricity and
magnetic fields,
Product marking, and
Safety.
Section 4--Software Standards: This section describes the design
and performance characteristics of the software embodied in voting
systems, addressing both system level software and voting system
application software. The requirements of this section are intended to
ensure that the overall objectives of accuracy, logical correctness,
privacy, system integrity, and reliability are achieved. Although this
section emphasizes software, the software standards may influence
hardware design in some voting systems.
The requirements of this section apply to all software developed
for use in voting systems, including:
Software provided by the voting system vendor and its
component suppliers, and
Software furnished by an external provider where the
software is potentially used in any way during voting system operation.
The general standards in this section apply to software used to
support the broad range of voting system activities, including pre-
voting, voting and post-voting activities. System specific Standards
are defined for ballot counting, vote processing, the creation of an
unalterable audit trail, and the generation of output reports and
files. Voting system software is also subject to the security
requirements of Section 6.
Section 5--Telecommunications Standards: This section describes the
requirements for the telecommunications components of voting systems.
Additionally, it defines the acceptable levels of performance against
these characteristics. For the purpose of the Standards,
telecommunications is defined as the capability to transmit and receive
data electronically regardless of whether the transmission is localized
within the polling place or the data is transmitted to a geographically
distinct location. The requirements in this section represent
functional and performance requirements for the transmission of data
that are used to operate the system and report official election
results. Where applicable, this section specifies minimum values for
critical performance and functional attributes involving
telecommunications hardware and software components.
This section addresses telecommunications hardware and software
across a broad range of technologies such as dial-up communications
technologies, high-speed telecommunications lines (public and private),
cabling technologies, communications routers, modems, modem drivers,
channel service units (CSU)/data service units (DSU), and dial-up
networking applications software.
Additionally, this section applies to voting-related transmissions
over public networks, such as those provided by regional telephone
companies and long distance carriers. This section also applies to
private networks regardless of whether the network is owned and
operated by the election jurisdiction. For systems that transmit data
over public networks, this section applies to telecommunications
components installed and operated at settings supervised by election
officials, such as polling places or central offices.
Section 6--Security Standards: This section starts with an overview
that provides a description of a new approach to securing voting
systems called independent dual verification. The overview introduces
the concept of independent dual verification and explains several
approaches for achieving it. Appendix D further explores independent
dual verification. Independent dual verification is not required in
VVSG Version 1, but will be required in Version 2. Following the
overview are 3 new sections describing requirements for voter verified
paper audit trails, wireless technology and software distribution and
setup. The remainder of the section is unchanged from VSS-2002 and
describes the security capabilities for a voting system, encompassing
the system's hardware, software, communications, and documentation. The
requirements of this section recognize that no predefined set of
security Standards will address and defeat all conceivable or
theoretical threats. However, the Standards articulate requirements to
achieve acceptable levels of integrity, reliability, and inviolability.
Ultimately, the objectives of the security Standards for voting systems
are to:
Establish and maintain controls that can ensure that
accidents, inadvertent mistakes, and errors are minimized,
Protect the system from intentional manipulation and
fraud,
Protect the system from malicious mischief,
Identify fraudulent or erroneous changes to the system,
and
Protect secrecy in the voting process.
These Standards are intended to address a broad range of risks to
the integrity of a voting system. While it is not possible to identify
all potential risks, the Standards identify several types of risk that
must be addressed, including:
Unauthorized changes to system capabilities for defining
ballot formats, casting and recording votes, calculating vote totals
consistent with defined ballot formats, and reporting vote totals,
Alteration of voting system audit trails,
Altering a legitimately cast vote,
Preventing the recording of a legitimately cast vote,
Introducing data for a vote not cast by a registered
voter,
Changing calculated vote totals,
Preventing access to vote data, including individual votes
and vote totals, to unauthorized individuals, and

[[Page 18932]]

Preventing access to voter identification data and data
for votes cast by the voter such that an individual can determine the
content of specific votes cast by the voter.
Section 7--Quality Assurance: In the Standards, quality assurance
is a vendor function with associated practices that confirms throughout
the system development and maintenance life-cycle that a voting system
conforms with the Standards and other requirements of state and local
jurisdictions. Quality assurance focuses on building quality into a
system and reducing dependence on system tests at the end of the life-
cycle to detect deficiencies.
This section describes the responsibilities of the voting system
vendor for designing and implementing a quality assurance program to
ensure that the design, workmanship, and performance requirements of
the Standards are achieved in all delivered systems and components.
These responsibilities include:
Development of procedures for identifying and procuring
parts and raw materials of the requisite quality, and for their
inspection, acceptance, and control.
Documentation of hardware and software development
processes.
Identification and enforcement of all requirements for in-
process inspection and testing that the manufacturer deems necessary to
ensure proper fabrication and assembly of hardware, as well as
installation and operation of software or firmware.
Procedures for maintaining all data and records required
to document and verify the quality inspections and tests.
Section 8--Configuration Management: This section contains specific
requirements for configuration management of voting systems. For the
purposes of the Standards, configuration management is defined as a set
of activities and associated practices that assures full knowledge and
control of the components of a system, beginning with its initial
development, progressing throughout its development and construction,
and continuing with its ongoing maintenance and enhancement. This
section describes activities in terms of their purpose and outcomes. It
does not describe specific procedures or steps to be employed to
accomplish them--these are left to the vendor to select.
The requirements of this section address a broad set of record
keeping, audit, and reporting activities that include:
Identifying discrete system components,
Creating records of formal baselines of all components,
Creating records of later versions of components,
Controlling changes made to the system and its components,
Submitting new versions of the system to Independent Test
Authorities (ITA)s,
Releasing new versions of the system to customers,
Auditing the system, including its documentation, against
configuration management records,
Controlling interfaces to other systems, and
Identifying tools used to build and maintain the system.
Vendors are required to submit documentation of these procedures to
the ITA as part of the Technical Data Package for system qualification
testing. Additionally, as articulated in state or local election laws,
regulations, or contractual agreements with vendors, authorized
election officials or their representatives reserve the right to
inspect vendor facilities and operations to determine conformance with
the vendor's reported configuration management procedures.
Section 9--Overview of Qualification Tests: This section provides
an overview for the qualification testing of voting systems.
Qualification testing is the process by which a voting system is shown
to comply with the requirements of the Standards and the requirements
of its own design and performance specifications. The testing also
evaluates the completeness of the vendor's developmental test program,
including the sufficiency of vendor tests conducted to demonstrate
compliance with stated system design and performance specifications,
and the vendor's documented quality assurance and configuration
management practices.
The qualification test process is intended to discover errors that,
should they occur in actual election use, could result in failure to
complete election operations in a satisfactory manner. This section
describes the scope of qualification testing, its applicability to
voting system components, documentation that is must be submitted by
the vendor, and the flow of the test process. This section also
describes differences between the test process for initial
qualification testing of a system and the testing for modifications and
re-qualification after a qualified system has been modified.
Since 1994, the testing described in this section has been
performed by an ITA that is certified by NASED. For the future, HAVA
provides for EAC-accredited testing authorities. HAVA tasks the
Director of NIST to assist the EAC by recommending laboratories for EAC
accreditation. NIST's National Voluntary Laboratory Accreditation
Program (NVLAP) is developing a program to evaluate competent
laboratories. While laboratories are being evaluated for recommendation
by the Director, testing will continue to be done by the ITAs
previously certified by NASED. The testing may be conducted by one or
more ITAs for a given system, depending on the nature of tests to be
conducted and the expertise of the certified ITA. The testing process
involves the assessment of, but is not limited to:
Absolute correctness of all ballot processing software,
for which no margin for error exists,
Operational accuracy in the recording and processing of
voting data, as measured by the error rate articulated in Volume I,
Section 3,
Operational failure or the number of unrecoverable
failures under conditions simulating the intended storage, operation,
transportation, and maintenance environments for voting systems, using
an actual time-based period of processing test ballots,
System performance and function under normal and abnormal
conditions, and
Completeness and accuracy of the system documentation and
configuration management records to enable purchasing jurisdictions to
effectively install, test, and operate the system.

Summary of Volume II Content

Section 1--Introduction: This section provides an overview of
Volume II, addressing the following topics:
Objectives of Volume II,
General contents of Volume II,
Qualification testing focus,
Qualification testing sequence,
Evolution of testing, and
Outline of contents.
Section 2--Technical Data Package: This section contains a
description of vendor documentation relating to the voting system that
shall be submitted with the system as a precondition for qualification
testing. These items are necessary to define the product and its method
of operation; to provide the vendor's technical and test data
supporting the its claims of the system's functional capabilities and
performance levels; and to document instructions and procedures
governing system operation and field maintenance. The content of the
Technical Data Package (TDP) shall contain a complete description of
the following information about the system:

[[Page 18933]]

Overall system design, including subsystems, modules, and
interfaces,
Specific functional capabilities,
Performance and design specifications,
Design constraints and compatibility requirements,
Personnel, equipment, and facilities necessary for system
operation, maintenance, and logistical support,
Vendor practices for assuring system quality during the
system's development and subsequent maintenance, and
Vendor practices for managing the configuration of the
system during development and for modifications to the system
throughout its life-cycle.
Section 3--Functionality Testing: This section contains a
description of the testing to be performed by the ITA to confirm the
functional capabilities of a voting system submitted for qualification
testing. It describes the scope and basis for functional testing, the
general sequence of tests within the overall test process, and provides
guidance on testing for accessibility. It also discusses testing of
functionality of systems that operate on personal computers.
Section 4--Hardware Testing: This section contains a description of
the testing to be performed by the ITAs to confirm the proper
functioning of the hardware components of a voting system submitted for
qualification testing. This section requires ITAs to design and perform
procedures that test the voting system hardware for both operating and
non-operating environmental tests. Hardware testing begins with non-
operating tests that require the use of an environmental test facility.
These are followed by operating tests that are performed partly in an
environmental facility and partly in a standard test laboratory or shop
environment. The non-operating tests are intended to evaluate the
ability of the system hardware to withstand exposure to various
environmental conditions incidental to voting system storage,
maintenance, and transportation. The procedures are based on test
methods contained in Military Standards (MIL-STD) 810D, modified where
appropriate, and include such tests as: Bench handling, vibration, low
and high temperature, and humidity.
The operating tests involve running the system for an extended
period of time under varying temperatures and voltages. This ensures
that the hardware meets or exceeds the minimum requirements for
reliability, data reading, and processing accuracy contained in Section
3 of Volume I. Although the procedure emphasizes equipment operability
and data accuracy, it is not an exhaustive evaluation of all system
functions. Moreover, the severity of the test conditions has in most
cases been reduced from that specified in the Military Standards to
reflect commercial, rather than military, practice.
Section 5--Software Testing: This section contains a description of
the testing to be performed by the ITAs to confirm the proper
functioning of the software components of a voting system submitted for
qualification testing. It describes the scope and basis for software
testing, the initial review of documentation to support software
testing, and the review of voting system source code.
The software qualification tests encompass a number of interrelated
examinations. The examinations include selective review of source code
for conformance with the vendor's stated standards, and other system
documentation provided by the vendor. The code inspection is
complemented by a series of functional tests to verify the proper
performance of all system functions controlled by the software.
Section 6--System Level Integration Testing: This section contains
a description of the testing conducted by the ITAs to confirm the
proper functioning of the fully integrated components of a voting
system submitted for qualification testing. It describes the scope and
basis for integration testing, testing of internal and external system
interfaces, testing of security capabilities, testing of accessibility
features, and the configuration audits, including the evaluation of
claims made in the system documentation.
System-level qualification tests address the integrated operation
of hardware, software and telecommunications capabilities (where
applicable) to assess the system's response to a range of both normal
and abnormal conditions in an attempt to compromise the system.
Section 7--Examination of Vendor Practices for Configuration
Management and Quality Assurance: This section contains a description
of examinations conducted by the ITAs to evaluate the extent to which
vendors meet the requirements for configuration management and quality
assurance. It describes the scope and basis for the examinations and
the general sequence of the examinations. It also provides guidance on
the substantive focus of the examinations.
In reviewing configuration management practices, the ITAs examine
the vendor's:
Configuration management policy,
Configuration identification policy,
Baseline, promotion and demotion procedures,
Configuration control procedures,
Release process and procedures, and
Configuration audit procedures.
In reviewing quality assurance practices, the ITAs examine the
vendor's:
Quality assurance policy,
Parts and materials tests and examinations,
Quality conformance plans, procedures and inspection
results, and
Voting system documentation.

Volume I, Section 1

Table of Contents

1 Introduction
1.1 Objectives and Usage of the Voting System Standards
1.2 Development History for Initial Standards
1.3 Update of the Standards
1.4 Accessibility for Individuals with Disabilities
1.5 Definitions
1.5.1 Voting System
1.5.2 Paper-Based Voting System
1.5.3 Direct Record Electronic (DRE) Voting System
1.5.4 Public Network Direct Record Electronic (DRE) Voting
System
1.5.5 Precinct Count Voting System
1.5.6 Central Count Voting System
1.6 Application of the Standards and Test Specifications
1.6.1 Qualification Tests
1.6.2 Certification Tests
1.6.3 Acceptance Tests
1.7 Conformance Clause
1.7.1 Scope and Applicability
1.7.2 Conformance Framework
1.7.2.1 Applicable entities
1.7.2.2 Relationship among entities
1.7.2.3 Conformance designations
1.7.3 Normative Language
1.7.4 Categorizing Requirements
1.7.5 Extensions
1.7.6 Implementation Statement
1.8 Outline of Contents

Introduction

1.1 Objectives and Usage of the Voting System Standards

State and local officials today are confronted with increasingly
complex voting system technology and an increased risk of voting system
failure. Responding to calls for assistance from the states, the United
States Congress authorized the Federal Election Commission (FEC) to
develop voluntary national voting systems standards for computer-based
systems. The resulting FEC Voting System Standards (``the

[[Page 18934]]

Standards'') seek to aid state and local election officials in ensuring
that new voting systems are designed to function accurately and
reliably, thus ensuring the system's integrity. States are free to
adopt the Standards in whole or in part. States may also choose to
enact stricter performance requirements for systems used in their
jurisdictions.
The Standards specify minimum functional requirements, performance
characteristics, documentation requirements, and test evaluation
criteria. For the most part, the Standards address what a voting system
should reliably do, not how system components should be configured to
meet these requirements. It is not the intent of the Standards to
impede the design and development of new, innovative equipment by
vendors. Furthermore, the Standards balance risk and cost by requiring
voting systems to have essential, but not excessive, capabilities.
The Standards are not intended to define appropriate election
administration practices. However, the total integrity of the election
process can only be ensured if implementation of the Standards is
coupled with effective election administration practices.
The Standards are intended for use by multiple audiences to support
their respective roles in the development, testing, and acquisition of
voting systems:
Authorities responsible for the analysis and testing of
such systems in support of qualification and/or certification of
systems for purchase within a designated jurisdiction;
State and local agencies evaluating voting systems to be
procured within their jurisdictions; and
Designers and manufacturers of voting systems.

1.2 Development History for Initial Standards

Much of the groundwork for the Standards' development was laid by a
national study conducted in 1975 by the National Bureau of Standards,
now known as the National Institute of Standards and Technology (NIST).
This study was requested by the FEC's Office of Election
Administrator's predecessor, the Office of Federal Elections of the
General Accounting Office. The report, ``Effective Use of Computing
Technology in Vote-Tallying,'' made a number of recommendations bearing
directly on the Standards project. After analyzing computer-related
election problems encountered in the past, the report concluded that
one of the basic causes for these difficulties was the lack of
appropriate technical skill at the state and local level for developing
or implementing sophisticated and complex standards against which
voting system hardware and software could be tested.
Following the release of this report, Congress mandated that the
FEC, with the cooperation and assistance of the National Bureau of
Standards, study and report on the feasibility of developing
``voluntary engineering and procedural performance standards for voting
systems used in the United States.'' (2 U.S.C. 431 Note) The resulting
1983 study cited a substantial number of technical and managerial
problems that affected the integrity of the vote counting process. It
also asserted the need for a federal agency to develop national
performance standards that could be used as a tool by state and local
election officials in the testing, certification, and procurement of
computer-based voting systems. In 1984, Congress approved initial
funding for the Standards.
The FEC held a series of public hearings in developing the initial
Standards. State and local election officials, election system vendors,
technical consultants, and others reviewed drafts of the proposed
criteria. The FEC considered their many comments and made appropriate
revisions. Before final issuance, the FEC publicly announced the
availability of the latest draft of the Standards in the Federal
Register and requested that all interested parties submit final
comments. The FEC meticulously reviewed all responses to the notice and
incorporated corrections and suitable suggestions. Ultimately, the
final product was the result of considerable deliberation, close
consultation with election officials, and careful consideration of
comments from all interested parties.
In January 1990, the FEC issued the performance standards and
testing procedures for punchcard, marksense, and direct recording
electronic (DRE) voting systems. The Standards did not cover paper
ballot and mechanical lever systems because paper ballots are
sufficiently self-explanatory not to require technical standards and
mechanical lever systems are no longer manufactured or sold in the
United States. The FEC also did not incorporate requirements for
mainframe computer hardware because it was reasonable to assume that
sufficient engineering and performance criteria already governed the
operation of mainframe computers. However, vote tally software
installed on mainframes is covered by the Standards.

1.3 Update of the Standards

Today, over two-thirds of the States have adopted the Standards in
whole or in part. As a result, the voting systems marketed today are
dramatically improved. Election officials are better assured that the
voting systems they procure will work accurately and reliably. Voting
system failures are declining and now primarily involve pre-Standard
equipment, untested equipment configurations, or the mismanagement of
tested equipment. Overall, systems integrity and the election processes
have improved markedly.
However, advances in voting technology, legislative changes, and
the proliferation of electronic voting systems make an update of the
Standards necessary. The industry has been marked by widespread
integration of personal computer technology and non-mainframe servers
into DRE voting systems.
In addition, voting systems need to be responsive to the Americans
with Disabilities Act (ADA) of 1990 and guidelines developed to assist
in implementing the ADA.

1.4 Accessibility for Individuals With Disabilities

Voters and election officials who use voting systems represent a
broad spectrum of the population, and include individuals with
disabilities who may have difficulty using traditional voting systems.
In developing accessibility provisions for the Standards, the FEC
requested assistance from the Access Board, the federal agency in the
forefront of promulgating accessibility provisions. The Access Board
submitted technical standards designed to meet the diverse needs of
voters with a broad range of disabilities. The FEC has adopted the
entirety of the Access Board's recommendations and incorporated them
into the Standards. These recommendations comprise the bulk of the
accessibility provisions found in Section 2.2.7. Implementing these
provisions, however, will not entirely eliminate the need to
accommodate the needs of some disabled voters by human interface.
The FEC anticipates that during the lifetime of this version of the
Standards increased obligations will be placed upon election officials
at every jurisdictional level to provide voting equipment tailored to
meet the needs of voters with disabilities. To facilitate jurisdictions
in meeting accessibility needs, the Standards mandate that every voting
system incorporate some accessible voting capabilities. The

[[Page 18935]]

Standards also mandate that systems incorporating a DRE component meet
specific technological requirements. To do so, it is anticipated that a
vendor will have to either configure all of the system's voting
stations to meet the accessibility specifications or will have to
design a unique station that conforms to the accessibility requirements
and is part of the overall voting system configuration.
Under no circumstances should compliance with requirements for
accessibility be viewed as mutually exclusive from compliance with any
other provision of the Standards. If a voting system contains a machine
uniquely designed to meet the accessibility requirements, such a
machine will be tested for compliance with the accessibility
requirements, as well as for compliance with all of the DRE standards,
in order to ensure that an accessible machine does not unintentionally
abrogate the mandates of the Standards.

1.5 Definitions

The Standards contain terms describing function, design,
documentation, and testing attributes of equipment and computer
programs. Unless otherwise specified, the intended sense of technical
terms is that which is commonly used by the information technology
industry. In some cases terminology is specific to elections or voting
systems, and a glossary of those terms is contained in Appendix A.
Nontechnical terms not listed in Appendix A shall be interpreted
according to their standard dictionary definitions.
Additionally, the following terms are defined below:
Voting system;
Paper-based voting system;
Direct record electronic (DRE) voting system;
Public network direct record electronic (DRE) voting
system;
Precinct count voting system; and
Central count voting system.
1.5.1 Voting System
A voting system is a combination of mechanical, electromechanical,
or electronic equipment. It includes the software required to program,
control, and support the equipment that is used to define ballots; to
cast and count votes; to report and/or display election results; and to
maintain and produce all audit trail information. A voting system may
also include the transmission of results over telecommunication
networks.
Additionally, a voting system includes the associated documentation
used to operate the system, maintain the system, identify system
components and their versions, test the system during its development
and maintenance, maintain records of system errors and defects, and
determine specific changes made after system qualification. By
definition, this includes all documentation required in Section 9.4.
Traditionally, a voting system has been defined by the mechanism
the system uses to cast votes and further categorized by the location
where the system tabulates ballots. However, the Standards recognize
that as the industry develops unique solutions to various challenges
and as voting systems become more responsive to the needs of election
officials and voters, the rigid dichotomies between voting system types
may be blurred. Innovations that use a fluid understanding of system
types can greatly improve the voting system industry, but only if
controls are in place to monitor and control integrity through the
proper evaluation of the system brought for qualification.
As such, vendors that submit a system that integrates components
from more than one traditional system type or a system that includes
components not addressed in this Standard shall submit the results of
all beta tests of the new system. Vendors also shall submit a proposed
test plan to the appropriate independent test authority recognized by
the National Association of State Election Directors (NASED) to conduct
national qualification testing of voting systems. The Standards permit
vendors to produce or utilize interoperable components of a voting
system that are tested within the full voting system configuration.
1.5.2 Paper-Based Voting System
A Paper-Based Voting System, (referred to in the initial Standards
as a Punchcard and Marksense [P&M] Voting System) records votes, counts
votes, and produces a tabulation of the vote count from votes cast on
paper cards or sheets. A punchcard voting system allows a voter to
record votes by means of holes punched in designated voting response
locations. A marksense voting system allows a voter to record votes by
means of marks made by the voter directly on the ballot, usually in
voting response locations. Additionally, a paper based system may
record votes using other approaches whereby the voter's selections are
indicated by marks made on a paper ballot by an electronic input
device, as long as such an input device does not independently record,
store, or tabulate the voters selections.
1.5.3 Direct Record Electronic (DRE) Voting System
A Direct Record Electronic (DRE) Voting System records votes by
means of a ballot display provided with mechanical or electro-optical
components that can be activated by the voter; that processes data by
means of a computer program; and that records voting data and ballot
images in memory components. It produces a tabulation of the voting
data stored in a removable memory component and as printed copy. The
system may also provide a means for transmitting individual ballots or
vote totals to a central location for consolidating and reporting
results from precincts at the central location.
1.5.4 Public Network Direct Record Electronic (DRE) Voting System
A Public Network Direct Record Electronic (DRE) Voting System is an
election system that uses electronic ballots and transmits vote data
from the polling place to another location over a public network as
defined in Section 5.1.2. Vote data may be transmitted as individual
ballots as they are cast, periodically as batches of ballots throughout
the Election Day, or as one batch at the close of voting. For purposes
of the Standards, Public Network DRE Voting Systems are considered a
form of DRE Voting System and are subject to the standards applicable
to DRE Voting Systems. However, because transmitting vote data over
public networks relies on equipment beyond the control of the election
authority, the system is subject to additional threats to system
integrity and availability. Therefore, additional requirements
discussed in Section 5 and 6 apply.
The use of public networks for transmitting vote data must provide
the same level of integrity as other forms of voting systems, and must
be accomplished in a manner that precludes three risks to the election
process: Automated casting of fraudulent votes, automated manipulation
of vote counts, and disruption of the voting process such that the
system is unavailable to voters during the time period authorized for
system use.
1.5.5 Precinct Count Voting System
A Precinct Count Voting System is a voting system that tabulates
ballots at the polling place. These systems typically tabulate ballots
as they are cast and print the results after the close of polling. For
DREs, and for some paper-based systems, these systems provide

[[Page 18936]]

electronic storage of the vote count and may transmit results to a
central location over public telecommunication networks.
1.5.6 Central Count Voting System
A Central Count Voting System is a voting system that tabulates
ballots from multiple precincts at a central location. Voted ballots
are typically placed into secure storage at the polling place. Stored
ballots are transported or transmitted to a central counting place. The
systems produce a printed report of the vote count, and may produce a
report stored on electronic media.

1.6 Application of the Standards and Test Specifications

The Standards apply to all system hardware, software,
telecommunications, and documentation intended for use to:
Prepare the voting system for use in an election;
Produce the appropriate ballot formats;
Test that the voting system and ballot materials have been
properly prepared and are ready for use;
Record and count votes;
Consolidate and report results;
Display results on-site or remotely; and
Maintain and produce all audit trail information.
In general, the Standards define functional requirements and
performance characteristics that can be assessed by a series of defined
tests. Standards are mandatory requirements and are designated by use
of the term ``shall.''
Some voting systems use one or more readily available commercial
off-the-shelf (COTS) devices (such as card readers, printers, or
personal computers) or software products (such as operating systems,
programming language compilers, or database management systems). COTS
devices and software are exempted from certain portions of the
qualification testing process as defined herein, as long as such
products are not modified for use in a voting system.
Generally, voting systems are subject to the following three
testing phases prior to being purchased or leased:
Qualification tests;
State certification tests; and
State and/or local acceptance tests.
1.6.1 Qualification Tests
Qualification tests validate that a voting system meets the
requirements of the Standards and performs according to the vendor's
specifications for the system. Such tests encompass the examination of
software; the inspection and evaluation of system documentation; tests
of hardware under conditions simulating the intended storage,
operation, transportation, and maintenance environments; operational
tests to validate system performance and function under normal and
abnormal conditions; and examination of the vendor's system
development, testing, quality assurance, and configuration management
practices. Qualification tests address individual system components or
elements, as well as the integrated system as a whole.
Since 1994, qualification tests for voting systems have been
performed by Independent Test Authorities (ITAs) certified by the
National Association of State Election Directors (NASED). NASED has
certified an ITA for either the full scope of qualification testing or
a distinct subset of the total scope of testing. To date, ITAs have
been certified only for distinct subsets of testing. Upon the
successful completion of testing by an ITA, the ITA issues a
Qualification Test Report to the vendor and NASED. The qualification
test report remains valid for as long as the voting system remains
unchanged.
Upon receipt of test reports that address the full scope of
testing, NASED issues a Qualification Number that indicates the system
has been tested by certified ITAs for compliance with the Standards and
qualifies for the certification process of states that have adopted the
Standards. The Qualification Number applies to the system as a whole,
and does not apply to individual system components or untested
configurations.
After a system has completed qualification testing, further
examination of a system is required if modifications are made to
hardware, software, or telecommunications, including the installation
of software on different hardware. Vendors request review of
modifications by the appropriate ITA based on the nature and scope of
changes made and the scope of the ITA's role in NASED qualification.
The ITA will determine the extent to which the modified system should
be resubmitted for qualification testing and the extent of testing to
be conducted.
Generally, a voting system remains qualified under the standards
against which it was tested, as long as no modifications not approved
by an ITA are made to the system. However, if a new threat to a
particular voting system is discovered, it is the prerogative of NASED
to determine which qualified voting systems are vulnerable, whether
those systems need to be retested, and the specific tests to be
conducted. In addition, when new standards supersede the standards
under which the system was qualified, it is the prerogative of NASED to
determine when systems that were qualified under the earlier standards
will lose their qualification, unless they are tested to meet current
standards.
Among other things, qualification testing complements and evaluates
the vendor's developmental testing and beta testing. The ITA is
expected to evaluate the completeness of the vendor's developmental
test program, including the sufficiency of vendor tests conducted to
demonstrate compliance with the Standards as well as the system's
performance specifications. The ITA undertakes sample testing of the
vendor's test modules and also designs independent system-level tests
to supplement and check those designed by the vendor. Although some of
the qualification tests are based on those prescribed in the Military
Standards, in most cases the test conditions are less stringent,
reflecting commercial, rather than military, practice.
1.6.2 Certification Tests
Certification tests are performed by individual states, with or
without the assistance of outside consultants, to:
Confirm that the voting system presented is the same as
the one qualified through the Standards;
Test for the proper implementation of state-specific
requirements;
Establish a baseline for future evaluations or tests of
the system, such as acceptance testing or state review after
modifications have been made; and
Define acceptance tests.
Precise certification test scripts are not included in the
Standards, as they must be defined by the state, with its laws,
election practices, and needs in mind. However, it is recommended that
they not duplicate qualification tests, but instead focus on functional
tests and qualitative assessment to ensure that the system operates in
a manner that is acceptable under state law. If a voting system is
modified after state certification, it is recommended that States
reevaluate the system to determine if further certification testing is
warranted.
Certification tests performed by individual states typically rely
on information contained in documentation provided by the vendor for
system design, installation, operations, required facilities and
supplies, personnel support and other aspects of the voting system.
States and jurisdictions may define information and documentation
requirements additional to those defined in the

[[Page 18937]]

Standards. By design, the Standards, and qualification testing of
voting systems for compliance with the Standards, do not address these
additional requirements. However, qualification testing addresses all
capabilities of a voting system stated by the vendor in the system
documentation submitted to an ITA, including additional capabilities
that are not required by the Standards.
1.6.3 Acceptance Tests
Acceptance tests are performed at the state or local jurisdiction
level upon system delivery by the vendor to:
Confirm that the system delivered is the specific system
qualified by NASED and, when applicable, certified by the state;
Evaluate the degree to which delivered units conform to
both the system characteristics specified in the procurement
documentation, and those demonstrated in the qualification and
certification tests; and
Establish a baseline for any future required audits of the
system.
Some of the operational tests conducted during qualification may be
repeated during acceptance testing.

1.7 Conformance Clause

1.7.1 Scope and Applicability
The Voluntary Voting System Guidelines (VVSG) define requirements
for conformance of voting systems. Conformance is defined in terms of
requirements that voting system vendors claiming conformance to these
Guidelines shall meet. The VVSG also provides the framework,
procedures, and requirements that testing authorities responsible for
the qualification of voting systems shall follow in order to qualify a
voting system for EAC certification. The requirements and procedures in
the VVSG may also be used by States to certify voting systems. To
ensure that correct voting system software has been distributed without
modification, the VVSG includes requirements for a national software
repository. Finally, the VVSG provides guidance in the form of best
practices to voting officials. These best practices are not mandated
and are not subject to testing by testing authorities to qualify voting
systems. They are provided as adjuncts to the technical requirements
for voting systems in order to ensure the integrity of the voting
process and to assist States in properly setting up, deploying, and
operating voting systems.
The Voluntary Voting System Guidelines define the minimum
requirements for voting systems and the process of testing voting
systems. The guidelines are intended for use by:
1. Designers and manufacturers of voting systems,
2. Testing authorities responsible for the analysis and testing of
voting systems in support of qualification of systems for purchase
within a designated jurisdiction,
3. National software repositories, either maintained by the
National Institute of Standard and Technology (NIST) or other EAC
designated repository,
4. (Optionally) Voting officials, including election judges, poll
workers, ballot designers and officials responsible for the
installation, operation, and maintenance of voting machines, and
5. (Optionally) testing authorities responsible for the State
certification of voting systems.
Minimum requirements specified in these guidelines include:
Functional requirements,
Performance characteristics,
Documentation requirements,
Test evaluation criteria, and
Procedural requirements.
1.7.2 Conformance Framework
This section provides the framework in which conformance is
defined. It identifies the entities for which these guidelines apply,
the relationship among the various entities and these guidelines,
structure of requirements, and the terminology used to indicate
conformance.
1.7.2.1 Applicable Entities
The requirements, prohibitions, options, and guidance specified in
these guidelines apply to voting systems, voting system vendors,
testing authorities, and repositories.
In general, requirements for designers and manufacturers of voting
systems in these guidelines apply to all voting systems, unless
prefaced with explanatory narrative describing unique applicability.
Other terms in these guidelines shall be construed as synonymous with
``all voting systems.'' They are:
``all systems,''
``systems,''
``the system,''
``the voting system,'' and
``each voting system.''
The term ``voting system vendor'' imposes documentation or testing
requirements on voting systems, via the manufacturer or vendor. Other
terms in these guidelines shall be construed as synonymous with
``voting system vendor. They are:
``vendors,''
``the vendor,''
``manufacturer or vendor,''
``voting system designers,'' and
``implementer.''
The terms used to designate requirements and procedural guidelines
for testing authorities are indicated by referring to Independent
Testing Authority (ITA) and EAC accredited testing authority. Under
HAVA, ITAs have been replaced by EAC accredited testing authorities. In
these guidelines, EAC accredited testing authority and ITA shall be
considered equivalent. In addition, the National Association of State
Election Directors (NASED) activities specified in these guidelines
shall be performed by the Election Assistance Commission (EAC).
The term ``repository'' will be used to designate requirements
levied on the national software repository maintained at NIST or any
other EAC designated repository. The repository maintained at NIST is
called the National Software Reference Library (NSRL).
Guidance and best practices for voting officials are indicated by
the notation ``Best Practices for Voting Officials'' preceding the best
practice statement.
1.7.2.2 Relationship Among Entities
Although conformance is defined for voting systems, it is the
voting system vendor that needs to implement these requirements and
provide the necessary documentation with the system. In order to claim
conformance to the Voluntary Voting Systems Guidelines, the voting
system vendor shall satisfy the minimum requirements specified in the
VVSG, including implementation of functionality, prescribed software
coding and assurance practices, and preparation of the Technical Data
Package (TDP). In order to claim that a voting system is qualified, the
voting system vendor shall satisfy the requirements for qualification
testing and successfully complete the test campaign with an ITA/testing
authority.
An ITA/EAC accredited test authority shall satisfy the requirements
for conducting qualification testing. The ITA/EAC accredited test
authority may use an operational environment that is derived from the
VVSG best practice guidelines for voting officials as part of their
testing to ensure that the voting system can be configured and operated
in a secure and reliable manner according to the voting system vendor's
documentation and as specified by the VVSG. Additionally, the ITA/EAC
accredited test authority shall coordinate and deliver the requisite
documentation to the EAC and copies of voting system software to the
repository. Note that in the VVSG, these

[[Page 18938]]

requirements and the relationship between the ITA/EAC accredited test
authority and the certification authority is with NASED, not the EAC.
The EAC is assuming the responsibility for certification of voting
systems from NASED.
The VVSG provides guidance denoted as ``Best Practices for Voting
Officials.'' This guidance may be used to allow jurisdictions to
incorporate appropriate procedures to help ensure that their voting
systems are reliable, accessible, usable, and secure. Furthermore, this
guidance may be used in training and incorporated into written
procedures for properly conducting the election and operating voting
systems.
Figure 1 provides an illustration of these relationships.
[GRAPHIC] [TIFF OMITTED] TN12AP06.018

1.7.2.3 Structure of Requirements
Sections of this document that augment the VSS-2002, by either
replacing VSS-2002 sections or adding new sections, are indicated by
line numbers, footer information (i.e., New Material, date, etc.) at
the bottom of pages with new material, and hierarchically structured
requirements. Each requirement is numbered according to a hierarchical
scheme in which higher-level requirements (such as ``provide
accessibility for blind voters'') are supported by lower-level
requirements (``provide an audio-tactile interface''). Thus,
requirements are contained (i.e., nested) within other requirements. A
nested requirement or lower-level requirement is a `child' to its
`parent' or higher-level requirement.
Some of these requirements are directly testable and some are not.
The latter tend to be higher-level and are included because (1) they
are testable indirectly insofar as their lower-level, children
requirements are testable, and (2) they often provide the structure and
rationale for the lower-level requirements. Satisfying the lower-level
requirement will result in satisfying its higher-level `parent'
requirement.
1.7.2.4 Conformance Designations
A voting system conforms if all the mandatory requirements that
apply to the voting system are fulfilled. An implementation statement
(see Section 1.7.6) or similar mechanism is used to describe the
capabilities, features and optional functions that have been
implemented and are subject to conformance and qualification testing.
There is no concept of partial conformance, e.g., a voting system is
80% conforming.
1.7.3 Normative Language
The following keywords are used to convey conformance requirements.
Shall--to indicate a mandatory requirement to be followed
(implemented) in order to conform. Synonymous with ``is required to.''
Is prohibited--to indicate a mandatory requirement that
indicates something that is not permitted (allowed), in order to
conform. Synonymous with ``shall not.''
Should, Is encouraged--to indicate an optional recommended
action, one that is particularly suitable, without mentioning or
excluding others. Synonymous with ``is permitted and recommended.''
May--to indicate an optional, permissible action.
Synonymous with ``is permitted.''
Normative text is directly applicable to achieving conformance to
this document. Informative parts of this document include examples,
extended explanations, and other matter that contain information
necessary for proper understanding of the VVSG and conformance to it.
Some sections in the VSSG have narrative text prefixed by the keywords:
Discussion or Best Practices for Voting Officials. This text is
informative and has no bearing on conformance.

[[Page 18939]]

1.7.4 Categorizing Requirements
In addition to defining a common set of requirements that apply to
all voting systems, the VVSG categorizes some requirements into related
groups of functionality to address equipment type, ballot tabulation
location, and voting system component (e.g., election management
system). Hence, not all requirements apply to all voting systems.
Specifically, if a category is not applicable to a voting system, then
the requirements in that category are not applicable. For example,
requirements categorized as ``DRE Systems'' (as in Volume I, Section
2.4.9) are not applicable to paper-based voting systems and thus are
ignored by paper-based systems.
Among the categories defined in the VVSG are two types of voting
systems with respect to mechanisms to cast votes--Paper-Based Voting
Systems and Direct Record Electronic (DRE) Voting Systems.
Additionally, voting systems are further categorized, in these
guidelines, by the locations where ballots are tabulated--Precinct
Count Voting Systems, which tabulate ballots at the polling place, and
Central Count Voting Systems, which tabulate ballots from multiple
precincts at a central location. The VVSG defines specific requirements
for systems that fall within these four categories as well as various
combinations of these categories.
Other categories for which requirements are defined include:
election management systems (EMS), methods of independent verification,
and telecommunication components.
1.7.5 Extensions
Extensions are additional functions, features, and/or capabilities
included in a voting system that are not required by the VVSG. To
accommodate the needs of States that may impose additional requirements
beyond those listed in these guidelines and to accommodate changes in
technology, these guidelines allow extensions. Thus, a voting system
may include extensions and still be conformant to the VVSG. The use of
extensions shall not contradict nor cause the nonconformance of
functionality defined in the VVSG.
1.7.6 Implementation Statement
An implementation statement provides information about a voting
system, by documenting the requirements that have been implemented by
the voting system. It can also be used to highlight optional features
and capabilities supported by the voting system, as well as to document
any extensions (i.e., additional functionality beyond what is required
in the standard). An implementation statement may take the form of a
checklist, to be completed for each voting system for which a claim of
conformance to the VVSG or subset of the VVSG is desired.
An implementation statement provides a concise summary and a quick
overview of requirements that have been implemented. The implementation
statement may also be used to identify the subset of a test suite that
would be applicable to the voting system being tested.
If an implementation statement is provided, it shall include
identifying information about the voting system, including at a minimum
versioning and date information. Additionally, a narrative description
of the voting system shall be included in the implementation statement.

1.8 Outline of Contents

The organization of the Standards has been simplified to facilitate
its use. Volume I, Voting System Performance Standards, is intended for
use by the broadest audience, including voting system developers,
equipment manufacturers and suppliers, independent test authorities,
local agencies that purchase and deploy voting systems, state
organizations that certify a system prior to procurement by a local
jurisdiction, and public interest organizations that have an interest
in voting systems and voting systems standards.
Section 2 describes the functional capabilities required
of voting systems.
Sections 3 through 6 describe specific performance
standards for election system hardware, software, telecommunications
and security, respectively.
Sections 7 and 8 describe practices for quality assurance
and configuration management, respectively, to be used by vendors, and
required information about vendor practices that will be reviewed in
concert with system qualification and certification test processes and
system purchase decisions.
Section 9 provides an overview of the test and measurement
process used by test authorities for qualification and re-qualification
of voting systems.
Appendix A provides a glossary of important terms used in
Volume I.
Appendix B lists the publications that were used for
guidance in the preparation of the Standards. These publications
contain information that is useful in interpreting and complying with
the requirements of the Standards.
Appendix C addresses issues of usability of voting
systems, commonly referred to as ``human factors.'' This appendix does
not represent mandates that voting systems will be tested against, but
rather contain recommendations and best practices on usability issues
designed to provide vendors and election officials with guidance on
designing and procuring systems that are easy and intuitive to use by
voters.
Volume II, Voting System Qualification Testing Standards describes
the standards for the technical information submitted by the vendor to
support testing; the development of test plans by the ITA for initial
system testing and testing of system modifications; the conduct of
system qualification tests by the ITA; and the test reports generated
by the ITA. This volume complements the content of Volume I and is
intended primarily for use by ITAs, state organizations that certify a
system, and vendors.

Volume I, Section 2

Table of Contents

2 Functional Capabilities
2.1 Scope
2.2 Overall System Capabilities
2.2.1 Security
2.2.2 Accuracy
2.2.2.1 Common Standards
2.2.2.2 DRE System Standards
2.2.3 Error Recovery
2.2.4 Integrity
2.2.4.1 Common Standards
2.2.4.2 DRE Systems Standards
2.2.5 System Audit
2.2.5.1 System Audit Purpose and Context
2.2.5.2 Operational Requirements
2.2.5.3 COTS General Purpose Computer System Requirements
2.2.6 Election Management System
2.2.7 Human Factors
2.2.7.1 Accessibility
2.2.7.2 Limited English Proficiency
2.2.7.3 Usability
2.2.7.4 Privacy
2.2.8 Vote Tabulating Program
2.2.8.1 Functions
2.2.8.2 Voting Variations
2.2.9 Ballot Counter
2.2.10 Telecommunications
2.2.11 Data Retention
2.3 Pre-Voting Functions
2.3.1 Ballot Preparation
2.3.1.1 General Capabilities
2.3.1.2 Ballot Formatting
2.3.1.3 Ballot Production
2.3.2 Election Programming
2.3.3 Ballot and Program Installation and Control
2.3.4 Readiness Testing
2.3.4.1 Common Standards
2.3.4.2 Paper-Based Systems
2.3.5 Verification at the Polling Place
2.3.6 Verification at the Central Location
2.4 Voting Functions
2.4.1 Opening the Polls
2.4.1.1 Opening the Polling Place (Precinct Count Systems)

[[Page 18940]]

2.4.1.2 Paper-Based System Standards
2.4.1.3 DRE System Standards
2.4.2 Activating the Ballot (DRE Systems)
2.4.3 Casting a Ballot
2.4.3.1 Common Standards
2.4.3.2 Paper-Based Systems Standards
2.4.3.3 DRE Systems Standards
2.5 Post-Voting Functions
2.5.1 Closing the Polling Place (Precinct Count)
2.5.2 Consolidating Vote Data
2.5.3 Producing Reports
2.5.3.1 Common Standards
2.5.3.2 Precinct Count Systems
2.5.4 Broadcasting Results
2.6 Maintenance, Transportation, and Storage

Functional Capabilities

2.1 Scope

This section contains standards detailing the functional
capabilities required of a voting system. This section sets out
precisely what it is that a voting system is required to do. In
addition, this section sets forth the minimum actions a voting system
must be able to perform to be eligible for qualification.
For organizational purposes, functional capabilities are
categorized by the phase of election activity in which they are
required:
[sdiam] Overall Capabilities: These functional capabilities apply
throughout the election process. They include security, accuracy,
integrity, system auditability, election management system, vote
tabulation, ballot counters, telecommunications, and data retention.
[sdiam] Pre-voting Capabilities: These functional capabilities are
used to prepare the voting system for voting. They include ballot
preparation, the preparation of election-specific software (including
firmware), the production of ballots or ballot pages, the installation
of ballots and ballot counting software (including firmware), and
system and equipment tests.
[sdiam] Voting Capabilities: These functional capabilities include
all operations conducted at the polling place by voters and officials
including the generation of status messages.
[sdiam] Post-voting Capabilities: These functional capabilities
apply after all votes have been cast. They include closing the polling
place; obtaining reports by voting machine, polling place, and
precinct; obtaining consolidated reports; and obtaining reports of
audit trails.
[sdiam] Maintenance, Transportation and Storage Capabilities: These
capabilities are necessary to maintain, transport, and store voting
system equipment.
In recognition of the diversity of voting systems, the Standards
apply specific requirements to specific technologies. Some of the
Standards apply only if the system incorporates certain optional
functions (for example, voting systems employing telecommunications to
transmit voting data). For each functional capability, common standards
are specified. Where necessary, common standards are followed by
standards applicable to specific technologies (i.e., paper-based or
DRE) or intended use (i.e., central or precinct count).

2.2 Overall System Capabilities

This section defines required functional capabilities that are
system-wide in nature and not unique to pre-voting, voting, and post-
voting operations. All voting systems shall provide the following
functional capabilities:
Security;
Accuracy;
Error recovery;
Integrity;
System auditability;
Election management system;
Accessibility:
Vote tabulating;
Ballot counters; and
Data Retention.
Voting systems may also include telecommunications components.
Technical standards for these capabilities are described in Sections 3
through 6 of the Standards.
2.2.1 Security
System security is achieved through a combination of technical
capabilities and sound administrative practices. To ensure security,
all systems shall:
a. Provide security access controls that limit or detect access to
critical system components to guard against loss of system integrity,
availability, confidentiality, and accountability.
b. Provide system functions that are executable only in the
intended manner and order, and only under the intended conditions.
c. Use the system's control logic to prevent a system function from
executing if any preconditions to the function have not been met.
d. Provide safeguards to protect against tampering during system
repair, or interventions in system operations, in response to system
failure.
e. Provide security provisions that are compatible with the
procedures and administrative tasks involved in equipment preparation,
testing, and operation.
f. If access to a system function is to be restricted or
controlled, the system shall incorporate a means of implementing this
capability.
g. Provide documentation of mandatory administrative procedures for
effective system security.
2.2.2 Accuracy
Memory hardware, such as semiconductor devices and magnetic storage
media, must be accurate. The design of equipment in all voting systems
shall provide for the highest possible levels of protection against
mechanical, thermal, and electromagnetic stresses that impact system
accuracy. Section 3 provides additional information on susceptibility
requirements.
2.2.2.1 Common Standards
To ensure vote accuracy, all systems shall:
a. Record the election contests, candidates, and issues exactly as
defined by election officials;
b. Record the appropriate options for casting and recording votes;
c. Record each vote precisely as indicated by the voter and be able
to produce an accurate report of all votes cast;
d. Include control logic and data processing methods incorporating
parity and check-sums (or equivalent error detection and correction
methods) to demonstrate that the system has been designed for accuracy;
and
e. Provide software that monitors the overall quality of data read-
write and transfer quality status, checking the number and types of
errors that occur in any of the relevant operations on data and how
they were corrected.
2.2.2.2 DRE System Standards
As an additional means of ensuring accuracy in DRE systems, voting
devices shall record and retain redundant copies of the original ballot
image. A ballot image is an electronic record of all votes cast by the
voter, including undervotes.
2.2.3 Error Recovery
To recover from a non-catastrophic failure of a device, or from any
error or malfunction that is within the operator's ability to correct,
the system shall provide the following capabilities:
a. Restoration of the device to the operating condition existing
immediately prior to the error or failure, without loss or corruption
of voting data previously stored in the device;
b. Resumption of normal operation following the correction of a
failure in a memory component, or in a data processing component,
including the central processing unit; and
c. Recovery from any other external condition that causes equipment
to become inoperable, provided that catastrophic electrical or
mechanical damage due to external phenomena has not occurred.

[[Page 18941]]

2.2.4 Integrity
Integrity measures ensure the physical stability and function of
the vote recording and counting processes.
2.2.4.1 Common Standards
To ensure system integrity, all systems shall:
a. Protect, by a means compatible with these Standards, against a
single point of failure that would prevent further voting at the
polling place;
b. Protect against the interruption of electronic power;
c. Protect against generated or induced electromagnetic radiation;
d. Protect against ambient temperature and humidity fluctuations;
e. Protect against the failure of any data input or storage device;
f. Protect against any attempt at improper data entry or retrieval;
g. Record and report the date and time of normal and abnormal
events;
h. Maintain a permanent record of all original audit data that
cannot be modified or overridden but may be augmented by designated
authorized officials in order to adjust for errors or omissions (e.g.
during the canvassing process.)
i. Detect and record every event, including the occurrence of an
error condition that the system cannot overcome, and time-dependent or
programmed events that occur without the intervention of the voter or a
polling place operator; and
j. Include built-in measurement, self-test, and diagnostic software
and hardware for detecting and reporting the system's status and degree
of operability.
2.2.4.2 DRE Systems Standards
In addition to the common standards, DRE systems shall:
a. Maintain a record of each ballot cast using a process and
storage location that differs from the main vote detection,
interpretation, processing, and reporting path; and
b. Provide a capability to retrieve ballot images in a form
readable by humans.
2.2.5 System Audit
This section describes the context and purpose of voting system
audits and sets forth specific functional requirements. Additional
technical audit requirements are set forth in Section 4.
2.2.5.1 System Audit Purpose and Context
Election audit trails provide the supporting documentation for
verifying the correctness of reported election results. They present a
concrete, indestructible archival record of all system activity related
to the vote tally, and are essential for public confidence in the
accuracy of the tally, for recounts, and for evidence in the event of
criminal or civil litigation.
The following audit trail requirements are based on the premise
that system-generated creation and maintenance of audit records reduces
the chance of error associated with manually generated audit records.
Because most audit capability is automatic, the system operator has
less information to track and record, and is less likely to make
mistakes or omissions.
The sections that follow present operational requirements critical
to acceptable performance and reconstruction of an election.
Requirements for the content of audit records are described in Section
4 of the Standards.
The requirements for all system types, both precinct and central
count, are described in generic language. Because the actual
implementation of specific characteristics may vary from system to
system, it is the responsibility of the vendor to describe each
system's characteristics in sufficient detail that ITAs and system
users can evaluate the adequacy of the system's audit trail. This
description shall be incorporated in the System Operating Manual, which
is part of the Technical Data Package (TDP).
Documentation of items such as paper ballots delivered and
collected, administrative procedures for system security, and
maintenance performed on voting equipment are also part of the election
audit trail, but are not covered in these technical standards. Future
volumes of the Standards will address these and other system operations
practices. In the interim, useful guidance is provided by the
Innovations in Election Administration #10, Ballot Security and
Accountability, available from the FEC's Office of Election
Administration.
2.2.5.2 Operational Requirements
Audit records shall be prepared for all phases of elections
operations performed using devices controlled by the jurisdiction or
its contractors. These records rely upon automated audit data
acquisition and machine-generated reports, with manual input of some
information. These records shall address the ballot preparation and
election definition phase, system readiness tests, and voting and
ballot-counting operations. The software shall activate the logging and
reporting of audit data as described in the following sections.
2.2.5.2.1 Time, Sequence, and Preservation of Audit Records
The timing and sequence of audit record entries is as important as
the data contained in the record. All voting systems shall meet the
following requirements for time, sequence and preservation of audit
records:
a. Except where noted, systems shall provide the capability to
create and maintain a real-time audit record. This capability records
and provides the operator or precinct official with continuous updates
on machine status. This information allows effective operator
identification of an error condition requiring intervention, and
contributes to the reconstruction of election-related events necessary
for recounts or litigation.
b. All systems shall include a real-time clock as part of the
system's hardware. The system shall maintain an absolute record of the
time and date or a record relative to some event whose time and data
are known and recorded.
c. All audit record entries shall include the time-and-date stamp.
d. The audit record shall be active whenever the system is in an
operating mode. This record shall be available at all times, though it
need not be continually visible.
e. The generation of audit record entries shall not be terminated
or altered by program control, or by the intervention of any person.
The physical security and integrity of the record shall be maintained
at all times.
f. Once the system has been activated for any function, the system
shall preserve the contents of the audit record during any interruption
of power to the system until processing and data reporting have been
completed.
g. The system shall be capable of printing a copy of the audit
record. A separate printer is not required for the audit record, and
the record may be produced on the standard system printer if all the
following conditions are met:
(1) The generation of audit trail records does not interfere with
the production of output reports;
(2) The entries can be identified so as to facilitate their
recognition, segregation, and retention; and
(3) The audit record entries are kept physically secure.
2.2.5.2.2 Error Messages
All voting systems shall meet the following requirements for error
messages:
a. The system shall generate, store, and report to the user all
error messages as they occur;

[[Page 18942]]

b. All error messages requiring intervention by an operator or
precinct official shall be displayed or printed unambiguously in easily
understood language text, or by means of other suitable visual
indicators;
c. When the system uses numerical error codes for trained
technician maintenance or repair, the text corresponding to the code
shall be self-contained, or affixed inside the unit device. This is
intended to reduce inappropriate reactions to error conditions, and to
allow for ready and effective problem correction;
d. All error messages for which correction impacts vote recording
or vote processing shall be written in a manner that is understandable
to an election official who possesses training on system use and
operation, but does not possess technical training on system servicing
and repair;
e. The message cue for all systems shall clearly state the action
to be performed in the event that voter or operator response is
required;
f. System design shall ensure that erroneous responses will not
lead to irreversible error; and
g. Nested error conditions shall be corrected in a controlled
sequence such that system status shall be restored to the initial state
existing before the first error occurred.
2.2.5.2.3 Status Messages
The Standards provide latitude in software design so that vendors
can consider various user processing and reporting needs. The
jurisdiction may require some status and information messages to be
displayed and reported in real-time. Messages that do not require
operator intervention may be stored in memory to be recovered after
ballot processing has been completed.
The system shall display and report critical status messages using
unambiguous indicators or English language text. The system need not
display non-critical status messages at the time of occurrence. Systems
may display non-critical status messages (i.e., those that do not
require operator intervention) by means of numerical codes for
subsequent interpretation and reporting as unambiguous text.
Systems shall provide a capability for the status messages to
become part of the real-time audit record. The system shall provide a
capability for a jurisdiction to designate critical status messages.
2.2.5.3 COTS General Purpose Computer System Requirements
Further requirements must be applied to COTS operating systems to
ensure completeness and integrity of audit data for election software.
These operating systems are capable of executing multiple application
programs simultaneously. These systems include both servers and
workstations (or ``PCs''), including the many varieties of UNIX and
Linux, and those offered by Microsoft and Apple. Election software
running on these COTS systems is vulnerable to unintended effects from
other user sessions, applications, and utilities, executing on the same
platform at the same time as the election software.
``Simultaneous processes'' of concern include unauthorized network
connections, unplanned user logins, and unintended execution or
termination of operating system processes. An unauthorized network
connection or unplanned user login can host unintended processes and
user actions, such as the termination of operating system audit, the
termination of election software processes, or the deletion of election
software audit and logging data. The execution of an operating system
process could be a full system scan at a time when that process would
adversely affect the election software processes. Operating system
processes improperly terminated could be system audit or malicious code
detection.
To counter these vulnerabilities, three operating system
protections are required on all such systems on which election software
is hosted. First, authentication shall be configured on the local
terminal (display screen and keyboard) and on all external connection
devices (``network cards'' and ``ports''). This ensures that only
authorized and identified users affect the system while election
software is running.
Second, operating system audit shall be enabled for all session
openings and closings, for all connection openings and closings, for
all process executions and terminations, and for the alteration or
deletion of any memory or file object. This ensures the accuracy and
completeness of election data stored on the system. It also ensures the
existence of an audit record of any person or process altering or
deleting system data or election data.
Third, the system shall be configured to execute only intended and
necessary processes during the execution of election software. The
system shall also be configured to halt election software processes
upon the termination of any critical system process (such as system
audit) during the execution of election software.
2.2.6 Election Management System
The Election Management System (EMS) is used to prepare ballots and
programs for use in casting and counting votes, and to consolidate,
report, and display election results. An EMS shall generate and
maintain a database, or one or more interactive databases, that enables
election officials or their designees to perform the following
functions:
a. Define political subdivision boundaries and multiple election
districts as indicated in the system documentation;
b. Identify contests, candidates, and issues
c. Define ballot formats and appropriate voting options;
d. Generate ballots and election-specific programs for vote
recording and vote counting equipment;
e. Install ballots and election-specific programs;
f. Test that ballots and programs have been properly prepared and
installed;
g. Accumulate vote totals at multiple reporting levels as indicated
in the system documentation;
h. Generate the post-voting reports required by Section 2.5; and
i. Process and produce audit reports of the data indicated in
Section 4.5.
2.2.7 Human Factors
The importance of human factors in the design of voting systems has
become increasingly apparent. It is not sufficient that the internal
operation of these systems be correct; in addition, voters and poll
workers must be able to use them effectively. There are some special
difficulties in the design of usable and accessible voting systems:
The voting task itself can be fairly complex; the voter
may have to navigate an electronic ballot, choose multiple candidates
in a single race or decide on abstrusely worded referenda.
Voting is performed infrequently, so learning and
familiarity are lower than for more frequent tasks, such as use of an
ATM.
Jurisdictions may change voting equipment, thus obviating
whatever familiarity the voter might have acquired.
Once the voting session has been completed by the voter,
there is never a chance for later correction.
Voting must be accessible to all eligible citizens,
whatever their age, physical abilities, language skills, or experience
with technology.
The challenge, then, is to provide a voting system and voting
environment that all voters can use comfortably, efficiently, and with
justified

[[Page 18943]]

confidence that they have cast their votes correctly. The requirements
within this section are intended to serve that goal.
Although there are many detailed requirements, three broad
principles motivate this section on human factors:
1. All Eligible and Potentially Eligible Voters Shall Have Access
to the Voting Process Without Discrimination.
The voting process shall allow eligible voters of whatever age,
condition, or background to be able to go through the entire voting
process with the same degree of independence, privacy, and confidence,
insofar as technology will allow. Note that the voting process includes
access to the polling place, instructions on how to vote, initiating
the voting session, choosing candidates, getting help as needed, review
of the ballot, VVPAT, if applicable, and final submission of the
ballot.
2. Each Cast Ballot Shall Capture the Intent of the Voter Who Cast
That Ballot.
Voters have the right to have the ballot presented to them in a
manner that is clear and usable. Voters should encounter no difficulty
or confusion in recording their choices.
3. The Voting Process Shall Preserve the Secrecy of the Ballot.
The voting process shall preclude anyone else from determining the
content of a voter's ballot, with or without the voter's cooperation.
If such a determination is made against the wishes of the voter, then
his or her privacy has been violated. The process must also preclude
the voter from disclosing the content of the ballot to anyone else.
All the requirements within Section 2.2.7 have the purpose of
improving the quality of interaction between voters and voting systems.
Requirements that are likely to be relevant only to those
with some disability are listed under Section 2.2.7.1, although they
may also assist those not usually described as having a disability,
e.g. voters with poor eyesight or somewhat limited dexterity.
Requirements that are likely to be relevant only to those
with limited English proficiency are listed in Section 2.2.7.2.
Finally, requirements for general usability make up
Section 2.2.7.3 and those for privacy, Section 2.2.7.4.
Certain abbreviations and terms are used extensively throughout
Section 2.2.7:
CIF: Common Industry Format: Refers to the format
described in ANSI/INCITS 354-2001 ``Common Industry Format (CIF) for
Usability Test Reports.''
Acc-VS: Accessible Voting Station--the voting station
equipped for individuals with disabilities referred to in HAVA
301(a)(3)(B).
ATI: Audio-Tactile Interface--a voter interface designed
so as not to require visual reading of a ballot. Audio is used to
convey information to the voter and sensitive tactile controls allow
the voter to convey information to the voting system.
ALVS: Alternative Language Voting Station--a voting
station designed to be usable by voters who have limited English
proficiency.
This section also uses common terms as defined in the updated
Glossary. Note in particular, the distinctions among ``voting system,''
``voting station,'' and ``voting process.''

1. The Voting Process Shall Be Accessible to Voters With Disabilities.
As a Minimum, Every Polling Place Shall Have at Least One Voting
Station Equipped for Individuals With Disabilities, as Provided in HAVA
301 (a)(3)(B). A Station So Equipped Is Referred to Herein as an
Accessible Voting Station (Acc-VS)

HAVA Section 301(a)(3) reads in part:

ACCESSIBILITY FOR INDIVIDUALS WITH DISABILITIES.--The voting
system shall--
(A) be accessible for individuals with disabilities, including
nonvisual accessibility for the blind and visually impaired, in a
manner that provides the same opportunity for access and
participation (including privacy and independence) as for other
voters;
(B) satisfy the requirement of subparagraph (A) through the use
of at least one direct recording electronic voting system or other
voting system equipped for individuals with disabilities at each
polling place;

The requirements within Section 2.2.7.1 are intended to address
this mandate. Ideally every voter would be able to vote independently
and privately. As a practical matter, there may be a small number of
voters whose disabilities are so severe that they will need personal
assistance. Nonetheless, the requirements of this section are meant to
make the voting system directly accessible to as many voters as
possible.
Note that this section does not replace requirements of other
sections, but adds to them. In particular, the requirements of Section
2.2.7.3 on usability apply either to all voting stations or, in some
cases, to all DRE voting stations; many of these requirements support
accessibility as well as general usability.
Certain accessibility features that are likely to be useful to a
wide range of voters are required on all voting stations, not just the
Acc-VS. Finally, note that the Acc-VS is not necessarily a full-fledged
DRE; for instance, an implementation may provide an ATI that generates
an optiscan ballot.
The outline for Section 2.2.7.1 is:

2.2.7.1 Accessibility
2.2.7.1.1 Voters with Disabilities--General
2.2.7.1.2 Vision
2.2.7.1.2.1 Partial Vision
2.2.7.1.2.2 Blind
2.2.7.1.3 Dexterity
2.2.7.1.4 Mobility
2.2.7.1.5 Hearing
2.2.7.1.6 Speech
2.2.7.1.7 Cognitive

1. The Voting Process Shall Incorporate Features That Are Applicable to
Several Types of Disability

Discussion: These features span the disability categories within
requirement 2.2.7.1 (e.g. vision, dexterity).

1.1 When the Provision of Accessibility Involves an Alternative Format
for Ballot Presentation, Then All the Other Information Presented to
Voters in the Case of Non-Disabled English-Literate Voters (Including
Instructions, Warnings, Messages, and Ballot Choices) Shall Also Be
Presented in That Alternative Format
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is a general principle to be followed for any
alternative format presentation. Two particular cases, (a) audio
formats and (b) non-English formats, are the subject of specific
requirements in later sections.
[Best Practice for Voting Officials] When the provision of
accessibility involves an alternative format for ballot presentation,
then all the other information presented to voters in the case of non-
disabled English-literate voters (including instructions, warnings,
messages, and ballot choices) is also presented in that alternative
format.

1.2 An Acc-VS Shall Provide Direct Accessibility Such That Voters'
Personal Assistive Devices Are Not Required for Voting
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Voters are not to be obliged to supply any special
equipment in order to vote. This requirement does not preclude the Acc-
VS from providing interfaces to assistive technology.

[[Page 18944]]

1.3 When the Primary Means of Voter Identification or Authentication
Uses Biometric Measures That Require a Voter To Possess Particular
Biological Characteristics, the Voting Process Shall Provide a
Secondary Means That Does Not Depend on Those Characteristics
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, if fingerprints were used for
identification, there would have to be another mechanism for voters
without usable fingerprints.
[Best Practice for Voting Officials] When the primary means of
voter identification or authentication uses biometric measures that
require a voter to possess particular biological characteristics, the
voting process provides a secondary means that does not depend on those
characteristics.
[Best Practice for Voting Officials] Polling places are subject to
the appropriate guidelines of the Americans with Disabilities Act (ADA)
of 1990 and of the Architectural Barriers Act (ABA) of 1968. This
requirement does not stem from HAVA, but rather is a reminder of other
legal obligations. For more details, see http://www.access-board.gov/ada-aba.htm and http://www.usdoj.gov/crt/ada/votingck.htm.

ck.htm.

2. The Voting Process Shall Be Accessible to Voters With Visual
Disabilities

Discussion: Note that all aspects of the voting process are to be
accessible, not just the voting station.

2.1 The Acc-VS Shall Be Accessible to Voters With Partial Vision

2.1.1 The Vendor Should Conduct Summative Usability Tests on the Acc-VS
Using Partially Sighted Subjects and Report the Test Results to the
Appropriate Testing Authority According to the Common Industry Format
(CIF)
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

Discussion: This requirement is meant to encourage Acc-VS designers
to conduct some realistic usability tests on the final product. For
now, it is purely a documentation recommendation. Future versions of
the VVSG will include requirements for usability testing to be
conducted by the testing authority, with specific performance
benchmarks.
2.1.2 The Acc-VS and Any Voting Station With an Electronic Image
Display Shall Be Capable of Showing All Information in at Least Two
Font Sizes, (a) 3.0-4.0 mm and (b) 6.3-9.0 mm, Under Control of the
Voter or Poll Worker
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: While larger font sizes may assist most voters with
poor vision, certain disabilities such as tunnel vision are best
addressed by smaller font sizes. It is anticipated that future versions
of the VVSG will require font size to be under the independent control
of the voter.
2.1.3 All Voting Stations Using Paper Ballots Should Make Provisions
for Voters With Poor Reading Vision
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Possible solutions include: (a) providing paper ballots
in at least two font sizes, 3.0-4.0 mm and 6.3-9.0 mm and (b) providing
a magnifying device.
2.1.4 An Acc-VS and Any Voting Station With a Black-and-White-Only
Electronic Image Display Shall Be Capable of Showing All Information in
High Contrast Either by Default or Under the Control of the Voter or
Poll Worker. High Contrast Is a Figure-to-Ground Ambient Contrast Ratio
for Text and Informational Graphics of at Least 6:1
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: It is anticipated that future versions of the VVSG will
require contrast to be under the independent control of the voter.
2.1.5 An Acc-Vs With a Color Electronic Image Display Shall Allow the
Voter or Poll Worker To Adjust the Color or the Figure-to-Ground
Ambient Contrast Ratio
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: See NASED Technical Guide 1 for examples of
how a voting station may meet this requirement by offering a limited
number of discrete choices. In particular, it is not required that the
station offer a continuous range of color or contrast values.
2.1.6 On All Voting Stations, the Default Color Coding Shall Maximize
Correct Perception by Voters and Operators With Color Blindness
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[Best Practice for Voting Officials] On all voting stations, the
default color coding maximizes correct perception by voters and
operators with color blindness.
2.1.7 On All Voting Stations, Color Coding Shall Not Be Used as the
Sole Means of Conveying Information, Indicating an Action, Prompting a
Response, or Distinguishing a Visual Element
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This implies that although color can be used for
emphasis, some other non-color mode must also be used to convey the
information, such as a shape or text style. For example, red can be
enclosed in an octagon shape.
2.1.8 Buttons and Controls on All Voting Stations Should Be
Distinguishable by Both Shape and Color
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The redundant cues have been found to be helpful to
those with partial vision.
2.1.9 Any Voting Station Using an Electronic Image Display Should Also
Provide Synchronized Audio Output To Convey the Same Information as
That on the Screen
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Synchronized presentation of information in both visual
and aural modes is a recommendation in this version of the VVSG, but it
is anticipated that this will become a requirement in future versions.

[[Page 18945]]

2.2 The Acc-VS Shall Be Accessible to Voters Who Are Blind

Discussion: Of course, many of the features under this requirement
are also useful for voters with partial vision (see requirement
2.2.7.1.2.1) and for voters who cannot read English for other
reasons (see requirement 2.2.7.2).
2.2.1 The Vendor Should Conduct Summative Usability Tests on the Acc-Vs
Using Subjects Who Are Blind and Report the Test Results to the
Appropriate Testing Authority According to the Common Industry Format
(CIF)
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

Discussion: This requirement is meant to encourage Acc-VS designers
to conduct some realistic usability tests on the final product. For
now, it is purely a documentation recommendation. Future versions of
the VVSG will include requirements for usability testing to be
conducted by the testing authority, with specific performance
benchmarks.
2.2.2 The Acc-VS Shall Provide an Audio-Tactile Interface (ATI) That
Supports the Full Functionality of a Normal Ballot Interface, as
Specified in Section 2.4
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Note the necessity of both audio output and tactilely
discernible controls for voter input. Full functionality includes at
least:
Instructions and feedback on initial activation of the
ballot (such as insertion of a smart card), if this is normally
performed by the voter on comparable voting stations,
Instructions and feedback to the voter on how to operate
the Acc-VS, including settings and options (e.g. volume control,
repetition),
Instructions and feedback for navigation of the ballot,
Instructions and feedback for voter selections in races
and referenda, including write-in candidates,
Instructions and feedback on confirming and changing
selections, and
Instructions and feedback on final submission of ballot.
2.2.2.1 The ATI of the Acc-VS Shall Provide the Same Capabilities To
Vote and Cast a Ballot as Are Provided by the Other Voting Stations or
by the Visual Interface of the Acc-VS. Therefore, Functional Features
That Exceed the Requirements of Section 2.4 Must Be Provided on a Non-
Discriminatory Basis
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, if a ``normal'' ballot supports voting a
straight party ticket and then changing the choice in a single race, so
must the ATI. This requirement is a special case of the more general
requirement 2.2.7.1.1.1.
2.2.2.2 The ATI Shall Allow the Voter To Have Any Information Provided
by the System Repeated
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.2.2.3 The ATI Shall Allow the Voter To Pause and Resume the Audio
Presentation
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.2.2.4 The ATI Shall Allow the Voter To Skip to the Next Contest or
Return to Previous Contests
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is analogous to the ability of sighted voters to
move on to the next race once they have made a selection or to abstain
from voting on a contest.
2.2.2.5 The ATI Should Allow the Voter To Skip Over the Reading of a
Referendum so as To Be Able To Vote on It Immediately
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is analogous to the ability of sighted voters to
skip over the wording of a referendum on which they have already made a
decision prior to the voting session (e.g. ``Vote yes on proposition
123''). It is anticipated that this recommendation will become
a requirement in future versions of the VVSG.
2.2.3 All Voting Stations That Provide Audio Presentation of the Ballot
Shall Conform to the Following Sub-Requirements
Discussion: These requirements apply to all audio output, not just
to the ATI of an Acc-VS.
2.2.3.1 The Ati Shall Provide Its Audio Signal Through an Industry
Standard Connector for Private Listening Using a 3.5Mm Stereo Headphone
Jack To Allow Voters To Use Their Own Audio Assistive Devices
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.2.3.2 When a Voting Station Utilizes a Telephone Style Handset/
Headset To Provide Audio Information, It Shall Provide a Wireless T-
Coil Coupling for Assistive Hearing Devices so as To Provide Access to
That Information for Voters With Partial Hearing. That Coupling Shall
Achieve at Least a Category T4 Rating as Defined by American National
Standard for Methods of Measurement of Compatibility Between Wireless
Communications Devices and Hearing Aids, ANSI C63.19
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.2.3.3 No Voting Station Shall Cause Electromagnetic Interference With
Assistive Hearing Devices That Would Substantially Degrade the
Performance of Those Devices. The Station, Considered as a Wireless
Device (WD) Shall Achieve at Least a Category T4 Rating as Defined by
American National Standard for Methods of Measurement of Compatibility
Between Wireless Communications Devices and Hearing Aids, ANSI C63.19
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: ``Hearing devices'' includes hearing aids and cochlear
implants.

[[Page 18946]]

2.2.3.4 A Sanitized Headphone or Handset Should Be Made Available to
Each Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This requirement can be achieved in various ways,
including the use of ``throwaway'' headphones, or of sanitary
coverings.
[Best Practice for Voting Officials] A sanitized headphone or
handset is made available to each voter.
2.2.3.5 The Voting Station Shall Set the Initial Volume for Each Voter
Between 40 and 50 dB SPL
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: A voter does not ``inherit'' the volume as set by the
previous user of the voting station.
2.2.3.6 The Voting Station Shall Provide a Volume Control With an
Adjustable Amplification From a Minimum of 20dB SPL Up to a Maximum of
105 dB SPL, in Increments No Greater Than 20dB
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.2.3.7 The Audio System Shall Be Able To Reproduce Frequencies Over
the Audible Speech Range Of 315 Hz To 10KHz
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.2.3.8 The Audio System Should Provide Information Via Recorded Human
Speech, Rather Than Synthesized Speech
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Most users prefer real speech over synthesized speech.
2.2.3.9 The Audio System Should Allow Voters To Control, Within
Reasonable Limits, the Rate of Speech
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Many blind voters are accustomed to interacting with
accelerated speech.
2.2.4 If the Normal Procedure Is To Have Voters Initialize the
Activation of the Ballot, the Acc-Vs Shall Provide Features That Enable
Voters Who Are Blind To Perform This Activation
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, smart cards might provide tactile cues so
as to allow correct insertion.
2.2.5 If the Normal Procedure Is for Voters To Submit Their Own
Ballots, Then the Voting Process Should Provide Features That Enable
Voters Who Are Blind To Perform This Submission
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, if voters normally feed their own optiscan
ballots into a reader, blind voters should also be able to do so.
[Best Practice for Voting Officials] If the normal procedure is for
voters to submit their own ballots, then the voting process provides
features that enable voters who are blind to perform this submission.
2.2.6 If the Normal Procedure Includes VVPAT, the Acc-VS Should Provide
Features That Enable Voters Who Are Blind To Perform This Verification
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, the Acc-VS might provide an automated
reader for the paper record that converts the contents of the paper
into audio output. It is anticipated that this recommendation will
become a requirement in future versions of the VVSG.
2.2.7 All Mechanically Operated Controls or Keys on an Acc-VS Shall Be
Tactilely Discernible Without Activating Those Controls or Keys
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.2.8 On an Acc-VS, the Status of All Locking or Toggle Controls or
Keys (Such as the ``Shift'' Key) Shall Be Visually Discernible, and
Discernible Either Through Touch or Sound
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3. The Voting Process Shall Be Accessible to Voters Who Lack Fine Motor
Control or the Use of Their Hands

3.1 The Vendor Should Conduct Summative Usability Tests on the Acc-VS
With Subjects Lacking Fine Motor Control and Report the Test Results to
the Appropriate Testing Authority According to the Common Industry
Format (CIF)
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

3.2 All Keys and Controls on the Acc-VS Shall Be Operable With One Hand
and Shall Not Require Tight Grasping, Pinching, or Twisting of the
Wrist. The Force Required To Activate Controls and Keys Shall Be No
Greater 5 lbs. (22.2 N)
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Controls are to be operable without excessive force.

3.3 The Acc-VS Controls Shall Not Require Direct Bodily Contact or for
the Body To Be Part of Any Electrical Circuit
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This requirement ensures that controls are operable by
individuals using prosthetic devices.

[[Page 18947]]

3.4 The Acc-VS Should Provide a Mechanism To Enable Non-Manual Input
That Is Functionally Equivalent to Tactile Input
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This recommendation ensures that the Acc-VS is operable
by individuals who do not have the use of their hands. All the
functionality of the Acc-VS (e.g. straight party voting, write-in
candidates) that is available through the other forms of input, such as
tactile, must also be available through the input mechanism if it is
provided by the Acc-VS.

4. The Voting Process Shall Be Accessible to Voters Who Use Mobility
Aids, Including Wheelchairs

4.1 The Acc-VS Shall Provide a Clear Floor Space of 30 Inches (760 mm)
Minimum by 48 Inches (1220 mm) Minimum for a Stationary Mobility Aid.
The Clear Floor Space Shall Be Level With No Slope Exceeding 1:48 and
Positioned for a Forward Approach or a Parallel Approach
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

[Best Practice for Voting Officials] The Acc-VS provides a clear
floor space of 30 inches (760 mm) minimum by 48 inches (1220 mm)
minimum for a stationary mobility aid. The clear floor space is level
with no slope exceeding 1:48 and positioned for a forward approach or a
parallel approach.

4.2 All Controls, Keys, Audio Jacks and Any Other Part of the Acc-VS
Necessary for the Voter To Operate the Voting System Shall Be Within
Reach as Specified Under the Following Sub-Requirements
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

Discussion: All dimensions are given in inches. To convert to
millimeters, multiply by 25.4 and then round to the nearest multiple of
5. Note that these sub-requirements have meaningful application mainly
to controls in a fixed location. A hand-held tethered control panel is
another acceptable way of providing reachable controls. All the sub-
requirements inherit the ``responsible entity'' and ``process''
properties.
[Best Practice for Voting Officials] All controls, keys, audio
jacks and any other part of the Acc-VS necessary for the voter to
operate the voting system are within the reach regions as specified in
the VVSG Volume I, Section 2.2.7.1.4.3.

4.2.1 If the Acc-VS Has a Forward Approach With No Forward Reach
Obstruction Then the High Reach Shall Be 48 Inches Maximum and the Low
Reach Shall Be 15 Inches Minimum. See Figure 2.2.7.1-1
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

4.2.2 If the Acc-VS Has a Forward Approach With a Forward Reach
Obstruction, the Following Sub-Requirements Apply. See Figure 2.2.7.1-2
4.2.2.1 The Forward Obstruction Shall Be No Greater Than 25 Inches in
Depth, Its Top No Higher Than 34 Inches and Its Bottom Surface No Lower
Than 27 Inches
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

4.2.2.2 If the Obstruction Is No More Than 20 Inches in Depth, Then the
Maximum High Reach Shall Be 48 Inches, Otherwise It Shall Be 44 Inches
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

4.2.2.3 Space Under the Obstruction Between the Finish Floor or Ground
and 9 Inches (230 mm) Above the Finish Floor or Ground Shall Be
Considered Toe Clearance and Shall Comply With the Following Sub-
Requirements
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

A. Toe clearance shall extend 25 inches (635 mm) maximum under the
obstruction.
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

B. The minimum toe clearance under the obstruction shall be either
17 inches (430 mm) or the depth required to reach over the obstruction
to operate the Acc-VS, whichever is greater.
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

C. Toe clearance shall be 30 inches (760 mm) wide minimum.

Voting System Vendor
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

4.2.2.4 Space Under the Obstruction Between 9 inches (230 mm) and 27
Inches (685 mm) Above the Finish Floor or Ground Shall Be Considered
Knee Clearance and Shall Comply With the Following Sub-Requirements
A. Knee clearance shall extend 25 inches (635 mm) maximum under the
obstruction at 9 inches (230 mm) above the finish floor or ground.
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

B. The minimum knee clearance at 9 inches (230 mm) above the finish
floor or ground shall be either 11 inches (280 mm) or 6 inches less
than the toe clearance, whichever is greater.
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

C. Between 9 inches (230 mm) and 27 inches (685 mm) above the
finish floor or ground, the knee clearance shall be permitted to reduce
at a rate of 1 inch (25 mm) in depth for each 6 inches (150 mm) in
height.
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

Discussion: It follows that the minimum knee clearance at 27 inches
above the finish floor or ground shall be 3 inches less than the
minimum knee clearance at 9 inches above the floor.
D. Knee clearance shall be 30 inches (760 mm) wide minimum.
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

[[Page 18948]]

4.2.3 If the Acc-VS Has a Parallel Approach With No Side Reach
Obstruction Then the Maximum High Reach Shall be 48 Inches and the
Minimum Low Reach Shall be 15 Inches. See Figure 2.2.7.1-3
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

4.2.4 If the Acc-VS Has a Parallel Approach With a Side Reach
Obstruction, the Following Sub-Requirements Apply. See Figure 2.2.7.1-4
4.2.4.1 The Side Obstruction Shall Be No Greater Than 24 Inches in
Depth and Its Top No Higher Than 34 Inches
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

4.2.4.2 If the Obstruction Is No More Than 10 inches in Depth, Then the
Maximum High Reach Shall Be 48 Inches, Otherwise It Shall Be 46 Inches
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

Discussion: Since this is a parallel approach, no clearance under
the obstruction is required.
4.2.5 All Labels, Displays, Controls, Keys, Audio Jacks, and Any Other
Part of the Acc-VS Necessary for the Voter To Operate the Voting System
Shall Be Easily Legible and Visible to a Voter in a Wheelchair With
Normal Eyesight (No Worse Than 20/40, Corrected) Who Is in an
Appropriate Position and Orientation with Respect to the Acc-VS
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: There are a number of factors that could make relevant
parts of the Acc-VS difficult to see: small lettering, controls and
labels tilted at an awkward angle from the voter's viewpoint, glare
from overhead lighting, etc.

5. The Voting Process Shall Be Accessible to Voters With Hearing
Disabilities

5.1 The Acc-VS Shall Incorporate the Features Listed Under Requirement
2.2.7.1.2.2.3 (Audio Presentation) To Provide Accessibility
to Voters With Hearing Disabilities
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Note especially the requirements for volume
initialization and control.
[Best Practice for Voting Officials] The Acc-VS incorporates the
features listed in the VVSG Volume I, Section 2.2.7.1.2.2.3 (audio
presentation) to provide accessibility to voters with hearing
disabilities.

5.2 If a Voting Station Provides Sound Cues as a Method To Alert the
Voter, the Tone Shall Be Accompanied by a Visual Cue
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For instance, the station might beep if the voter
attempts to overvote. If so, there would have to be an equivalent
visual cue, such as the appearance of an icon, or a blinking element.

6. The Voting Process Shall Be Accessible to Voters With Speech
Disabilities

6.1 No Voting Station Shall Require Voter Speech for its Operation
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This does not preclude a voting station from offering
speech input as an option, but speech must not be the only means of
input.

7. The Voting Process Should Be Accessible to Voters With Cognitive
Disabilities
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: At present there are no design features specifically
aimed at helping those with cognitive disabilities. Section
2.2.7.1.2.1.9, the synchronization of audio with the screen in a DRE,
is helpful for some cognitive disabilities such as dyslexia. Section
2.2.7.3.3 also contains some relevant guidelines.
[Best Practice for Voting Officials] The voting process is made
accessible to voters with cognitive disabilities.

[[Page 18949]]

[GRAPHIC] [TIFF OMITTED] TN12AP06.012

2. The Voting Process Shall Be Accessible to Voters Who Are Not Fully
Literate in English. This Requirement May Be Satisfied by Providing
Voting Stations in a Polling Place That Accommodate Those Without a
Full Command of English. See HAVA 301 (a)(4) and 241 (b)(5). Such a
Facility is Referred to Herein as an Alternative Language Voting
Station (ALVS)

HAVA Section 301 (a)(4) reads:

ALTERNATIVE LANGUAGE ACCESSIBILITY.--The voting system shall
provide alternative language accessibility pursuant to the
requirements of section 203 of the Voting Rights Act of 1965 (42
U.S.C. 1973aa-1a).

The requirements within Section 2.2.7.2 are intended to address
this mandate. Ideally every voter would be able to vote independently
and privately, regardless of language. As a practical matter,
alternative language access is mandated under the Voting Rights Act of
1975, subject to certain thresholds, e.g. if the language group exceeds
5% of the voting age citizens.
Note that the provision of an audio interface for people with
visual disabilities as described in Section 2.2.7.1 may also assist
voters who speak English, but are unable to read it.
The outline for section 2.2.7.2 is:

2.2.7.2. Alternative Languages
2.2.7.2.1 Complete Information

[[Page 18950]]

2.2.7.2.2 Spelling of Names
2.2.7.2.3 Literate Voters
2.2.7.2.4 Illiterate Voters

1. All the Information Presented in the Normal Case of English-literate
Voters (Including Instructions, Warnings, Messages, and Ballot Choices)
Shall Also Be Presented by the ALVS, Whether the Language Is Written or
Spoken
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is in keeping with general requirement
2.2.7.1.1.1.

2. Regardless of the Language, Candidate Names Shall Be Displayed or
Pronounced in English on All Ballots. For Written Languages That Do Not
Use Roman Characters (e.g. Chinese, Japanese, Korean, Arabic), the
Ballot Shall Include Transliteration of Candidate Names Into the
Relevant Language
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[Best Practice for Voting Officials] Regardless of the language,
candidate names are displayed or pronounced in English on all ballots.
For written languages that do not use Roman characters (e.g., Chinese,
Japanese, Korean, Arabic), the ballot includes transliteration of
candidate names into the relevant language.

3. For Literate Voters, the ALVS Shall Provide Printed or Displayed
Instructions, Messages, and Ballots in Their Preferred Language,
Consistent With State and Federal Law
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.1 The Vendor Should Conduct Summative Usability Tests on the ALVS
With Literate Subjects Who Neither Speak Nor Read English and Report
the Test Results According to the Common Industry Format (CIF)
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

4. For Illiterate Voters, the ALVS Shall Provide Spoken Instructions
and Ballots in the Preferred Language of the Voter, Consistent With
State and Federal Law. The Requirements and Sub-Requirements of
2.2.7.1.2.2.2 (Acc-VS/ATI) Shall Apply to This Mode of
Interaction
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Note that some languages have no widely accepted
written form.

3. The Voting Process Shall Provide a High Level of Usability to the
Voters. Accordingly, Voters Shall Be Able to Negotiate the Process
Effectively, Efficiently, and Comfortably

Discussion: The first Voting System Standards codified in HAVA
relate to the interaction between the voter and the voting system. HAVA
Section 301 begins:
SEC. 301. VOTING SYSTEMS STANDARDS.
a. Requirements.--Each voting system used in an election for
Federal office shall meet the following requirements:
1. In general.--
A. Except as provided in subparagraph (B), the voting system
(including any lever voting system, optical scanning voting system, or
direct recording electronic system) shall--
i. Permit the voter to verify (in a private and independent manner)
the votes selected by the voter on the ballot before the ballot is cast
and counted;
ii. Provide the voter with the opportunity (in a private and
independent manner) to change the ballot or correct any error before
the ballot is cast and counted (including the opportunity to correct
the error through the issuance of a replacement ballot if the voter was
otherwise unable to change the ballot or correct any error); and
iii. If the voter selects votes for more than one candidate for a
single office--
I. Notify the voter that the voter has selected more than one
candidate for a single office on the ballot;
II. Notify the voter before the ballot is cast and counted of the
effect of casting multiple votes for the office; and
III. Provide the voter with the opportunity to correct the ballot
before the ballot is cast and counted.
B. A State or jurisdiction that uses a paper ballot voting system,
a punch card voting system, or a central count voting system (including
mail-in absentee ballots and mail-in ballots), may meet the
requirements of subparagraph (A)(iii) by--
i. Establishing a voter education program specific to that voting
system that notifies each voter of the effect of casting multiple votes
for an office; and
ii. Providing the voter with instructions on how to correct the
ballot before it is cast and counted (including instructions on how to
correct the error through the issuance of a replacement ballot if the
voter was otherwise unable to change the ballot or correct any error).
C. The voting system shall ensure that any notification required
under this paragraph preserves the privacy of the voter and the
confidentiality of the ballot.''
The requirements of this section supplement these basic HAVA
mandates and also HAVA's support for improved usability (see Section
243 and Section 221(e)(2)(D)).
Voting and Usability
Usability is defined generally as a measure of the effectiveness,
efficiency, and satisfaction achieved by a specified set of users with
a given product in the performance of specified tasks. In the context
of voting, the primary users are the voters (but also poll workers),
the product is the voting system, and the task is the correct
representation of one's choices in the election. Additional
requirements for task performance are independence and privacy: the
voter should normally be able to complete the voting task without
assistance from others (although the voting system itself may offer
help), and the voter's choices should be private (see Section 2.2.7.4).
Aside from its intrinsic undesirability, lack of independence or
privacy may adversely affect effectiveness (e.g. by possibly inhibiting
the voter's free choice) and efficiency (e.g. by slowing down the
process).
Among the ``bottom-line'' metrics for usability are:
low error rate for marking the ballot (the voter's
intention is correctly conveyed to and represented within the voting
system),
efficient operation (time required to vote is not
excessive), and
satisfaction (voter experience is safe, comfortable, free
of stress, and instills confidence).
These criteria define the core of good voting system usability. The
purpose of the detailed requirements listed below is to help voting
systems meet the core criteria.

[[Page 18951]]

Methodology for Requirements
It is the intention of the TGDC that in forthcoming versions of the
VVSG, usability will be addressed by high-level performance-based
requirements. That is, the requirements will directly address metrics
for effectiveness (e.g. correct capture of voters' intentions),
efficiency (e.g. time taken to vote), and satisfaction. Until the
supporting research is completed, however, the contents of this
subsection are limited to a somewhat basic set of widely accepted
design requirements and lower-level performance requirements. The
reasons for this approach are:
These are to serve as interim requirements, pending the
issuance of high-level performance requirements.
The actual benefit of numerous detailed design guidelines
is difficult to prove or measure.
The technical complexity and costs of a large set of
detailed requirements may not be justified.
Guidelines that are difficult to test because of
insufficient specificity have been omitted.
This is not to say that an extensive set of design guidelines is
without value. But we wish to distinguish between good advice to be
considered by developers and strict requirements that will be enforced
by a regime of formal testing. For more detail on the issue of design
vs. performance standards, see Sections 2.3 and 6.1 et al. of NIST
Special Publication 500-256: Improving the Usability and Accessibility
of Voting Systems and Products (http://vote.nist.gov/ Final%20Human%

20Factors%20 Report%20% 205-04.pdf).
General Issues for the Usability Requirements
As mentioned in Section 2.2.7.1, many of the guidelines in this
section enhance accessibility as well as general usability.
The scope of usability includes the entire voting process, although
the emphasis herein is on the interface between the voter and the
voting station.
The requirements in this sub-section generally assume a visual-
tactile interface, but also see requirements in Sections 2.2.7.1 and
Section 2.2.7.2 for alternative formats, including audio.
The outline for Section 2.2.7.3 is:
2.2.7.3 Usability
2.2.7.3.1 Usability Testing by Vendor
2.2.7.3.2 Functional Capabilities
2.2.7.3.3 Cognitive Issues
2.2.7.3.4 Perceptual Issues
2.2.7.3.5 Interaction Issues

1. The Vendor Should Conduct Summative Usability Tests on the Voting
System Using Subjects Representative of the General Population and
Report the Test Results to the Appropriate Testing Authority According
to the Common Industry Format (CIF)
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

2. The Voting Process Shall Provide Certain Functional Capabilities To
Support Voter Usability

2.1 As Mandated by HAVA 301(a)(1)(A), the Voting System Shall Support a
Process That Allows the Voter To Review His or Her Completed Ballot
Before Final Submission in Order To Verify That it Correctly Represents
the Intended Vote and To Correct the Ballot if Mistakes Are Detected
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Note that this review and correction may be achieved by
procedural means (e.g. in the case of paper ballots), as well as
technical (see HAVA 301(a)(1)(B)). This requirement is a brief
paraphrase of the HAVA language but of course the statutory language is
determinative.

2.2 As Mandated by HAVA 301(a)(1)(A), the Voting System Shall Support a
Process That Notifies the Voter if He or She Has Attempted To Vote for
More Candidates Than the Maximum Permitted in a Given Race and That
Provides the Voter With the Opportunity To Correct the Ballot Before
Final Submission
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Note that this notification and correction may be
achieved by procedural means (e.g. in the case of paper ballots), as
well as technical (see HAVA 301(a)(1)(B)). This requirement is a brief
paraphrase of the HAVA language but of course the statutory language is
determinative.

2.3 DRE Voting Stations Shall Allow the Voter To Change a Vote Within a
Race Before Advancing to the Next Race
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The point here is that voters using a DRE should not
have to wait for the final ballot review in order to change a vote.

2.4 The Voting System Shall Support a Process That Notifies the Voter
if He or She Has Attempted To Vote for Fewer Candidates Than the
Maximum Permitted in a Given Race and That Provides the Voter With the
Opportunity To Change the Ballot Before Final Submission. The Process
Shall Also Notify the Voter That Such an ``Undervote'' Is Permitted and
Shall Accept a Ballot if the Voter so Chooses
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Note that this notification and correction may be
achieved by procedural means (e.g. in the case of paper ballots), as
well as technical (see HAVA 301(a)(1)(B)).

2.5 DRE Voting Stations Should Provide Navigation Controls That Allow
the Voter To Advance to the Next Race or Go Back to the Previous Race
Before Completing a Vote on the Race or Races Currently Being Presented
(Whether Visually or Aurally)
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, the voter should not be forced to proceed
sequentially through all the races and/or candidates before going back
to check the status of a previous race.

[[Page 18952]]

3. The Voting Process Shall Be Designed To Minimize Cognitive
Difficulties for the Voter

3.1 Consistent With Election Law, the Voting System Should Support a
Process That Does Not Introduce Any Bias for or Against Any of the
Choices To Be Made by the Voter. In Both Visual and Aural Formats,
Candidates and Choices Shall Be Presented in an Equivalent Manner
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Certain differences in presentation are unavoidable,
such as the order in which candidates are listed, and write-in
candidates are inherently more difficult to vote for. But comparable
characteristics such as font size or voice volume and speed must be the
same for all choices.
3.2 The Voting System or Related Materials Shall Provide Clear
Instructions and Assistance so as To Allow Voters To Successfully
Execute and Cast Their Ballots Independently
Discussion: Voters should not routinely need to ask for human
assistance.
3.2.1 Voting Stations or Related Materials Shall Provide a Means for
the Voter To Get Help at Any Time During the Voting Session
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voter should always be able to get help at the
station if confused. DRE voting stations may provide this with a
distinctive ``help'' button. Any type of voting station may provide
written instructions that are available and separate from the ballot.
Note special requirements for the Acc-VS in requirement
2.2.7.1.2.2.2 (Acc-VS/ATI).
3.2.2 The Voting Station Shall Provide Instructions for All Its Valid
Operations
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If an operation is available to the voter, it must be
documented. Examples include how to change a vote, how to navigate
among races, how to cast a party-line vote, and how to cast a write-in
vote.

3.3 The Voting System Shall Provide the Capability To Design a Ballot
for Maximum Clarity and Comprehension

3.3.1 The Voting Station Should Not Visually Present a Single Race
Spread Over Two Pages or Two Columns
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Such a visual separation poses the risk that the voter
will perceive the race as two races. Of course, if a race has a very
large number of candidates, it may be infeasible to observe this
guideline.
[Best Practice for Voting Officials] The voting station does not
visually present a single race spread over two pages or two columns.
3.3.2 The Ballot Shall Clearly Indicate the Maximum Number of
Candidates for Which One Can Vote Within a Single Race
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[Best Practice for Voting Officials] The ballot clearly indicates
the maximum number of candidates for which one can vote within a single
race.
3.3.3 There Shall Be a Consistent Relationship Between the Name of a
Candidate and the Mechanism Used to Vote for That Candidate
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, if the response field where voters
indicate their selections is located to the left of a candidate's name,
then each response field shall be located to the left of the associated
candidate's names.
[Best Practice for Voting Officials] The ballot presents the
relationship between the name of a candidate and the mechanism used to
vote for that candidate in a consistent manner.

3.4 Warnings and Alerts Issued by the Voting Station Should Clearly
State the Nature of the Problem and the Set of Responses Available to
the Voter. The Warning Should Clearly State Whether the Voter Has
Performed or Attempted an Invalid Operation or Whether the Voting
Equipment Itself Has Failed in Some Way
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: In case of an equipment failure, the only action
available to the voter might be to get assistance from a poll worker.

3.5 The Use of Color by the Voting Station Should Agree With Common
Conventions: (a) Green, Blue or White Is Used for General Information
or as a Normal Status Indicator; (b) Amber or Yellow Is Used to
Indicate Warnings or a Marginal Status; (c) Red Is Used to Indicate
Error Conditions or a Problem Requiring Immediate Attention
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

4. The Voting Process Shall Be Designed to Minimize Perceptual
Difficulties for the Voter

4.1 No Display Screen of a Voting Station Shall Flicker With a
Frequency Between 2 Hz and 55 Hz
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Aside from usability concerns, this requirement
protects voters with epilepsy.

4.2 Any Aspect of the Voting Station That is Adjustable by the Voter or
Poll Worker, Including Font Size, Color, Contrast, and Audio Volume,
Shall Automatically Reset to a Standard Default Value Upon Completion
of That Voter's Session
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This implies that the voting station presents the same
initial appearance to every voter (excluding, of course, substantive
differences in the ballot content due to residence or party of the
voter).

[[Page 18953]]

4.3 If Any Aspect of a Voting Station is Adjustable by the Voter, There
Should Be a Mechanism to Reset All Such Aspects to Their Default Values
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The purpose is to allow a voter who has adjusted the
station into an undesirable state to reset all the aspects so as to get
a fresh start.

4.4 The Minimum Font Size for All Text Intended for the Voter During
the Voting Session Shall Be 3.0mm (Measured as the Height of a Capital
Letter)
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

4.5 All Text Intended for the Voter During the Voting Session Should Be
Presented in a Sans Serif Font
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Experimentation has shown that users prefer such a font
and the legibility of serif and sans serif fonts is equivalent.

4.6 The Minimum Figure-to-Ground Ambient Contrast Ratio for All Text
and Informational Graphics (Including Icons) Intended for the Voter
Shall Be 3:1
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5. The Voting Process Shall Be Designed to Minimize Interaction
Difficulties for the Voter

5.1 Voting Stations With Electronic Image Displays Shall Not Require
Page Scrolling by the Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is not an intuitive operation for those unfamiliar
with the use of computers. Even those experienced with computers often
do not notice a scroll bar and miss information below the page. DREs
may require voters to move to the next or previous ``page.''

5.2 The Voting Station Shall Provide Unambiguous Feedback Regarding the
Voter's Selection, Such as Displaying a Checkmark Beside the Selected
Option or Conspicuously Changing Its Appearance
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.3 If the Voting Station Requires a Response by a Voter Within a
Specific Period of Time, It Shall Issue an Alert at Least 20 Seconds
Before This Time Period Has Expired and Provide a Means by Which the
Voter May Receive Additional Time
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.4 Input Mechanisms Shall Be Designed so as to Minimize Accidental
Activation (Also, See Requirement 2.2.7.1.2.2.7 on Tactile
Discernability)

5.4.1 On Touch Screens, the Sensitive Touch Areas Shall Have a Minimum
Height of 0.5 Inches and Minimum Width of 0.7 Inches. The Vertical
Distance Between the Centers of Adjacent Areas Shall Be at Least 0.6
Inches, and the Horizontal Distance at Least 0.8 Inches
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.4.2 No Key or Control on a Voting Station Shall Have a Repeat Feature
Enabled
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is to preclude accidental activation.

4. The Voting Process Shall Preclude Anyone Else From Determining the
Content of a Voter's Ballot, With or Without the Voter's Cooperation

Discussion: Voter privacy is strongly supported by HAVA--see
Sections 221(e)(2)(C) and 301(a)(1). In this subsection, we address
only privacy concerns in relation to human factors issues, but not with
respect to the processing of cast ballots.
Although elections in American history have sometimes been public
(and certain ``town-hall'' questions are still voted openly), the use
of the secret ballot for political office is now universal.
Privacy in this context, including the property of the voter being
unable to disclose his or her vote, ensures that the voter can make
choices based solely on his or her own preferences without intimidation
or inhibition. Among other practices, this forbids the issuance of a
receipt to the voter that would provide proof to another how he or she
voted.
The outline for Section 2.2.7.4 is:

2.2.7.4 Privacy
2.2.7.4.1 Privacy at the polling place
2.2.7.4.2 No preservation of alternative formats
2.2.7.4.3 Absentee Balloting

1. The Voting Station and Polling Place Shall Be Configured so as to
Prevent Others From Learning the Contents of a Voter's Ballot

1.1 The Ballot and Any Input Controls Shall Be Visible Only to the
Voter During the Voting Session and Ballot Submission
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[Best Practice for Voting Officials] The ballot and any input
controls are visible only to the voter during the voting session and
ballot submission. Poll workers need to take into account such factors
as visual barriers, windows, permitted waiting areas for other voters,
and procedures for ballot submission when not performed at the voting
station, e.g. submission of optiscan ballots to a central reader.

1.2 The Audio Interface Shall Be Audible Only to the Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Voters who are hard of hearing but need to use an audio
interface may also need to increase the volume of the audio. Such
situations require headphones with low sound leakage.
[Best Practice for Voting Officials] The audio interface is audible
only to the voter.

[[Page 18954]]

1.3 As Mandated By HAVA 301(a)(1)(C), the Voting System Shall Notify
the Voter of an Attempted Overvote in a Way That Preserves the Privacy
of the Voter and the Confidentiality of the Ballot
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This requirement is a brief paraphrase of the HAVA
language but of course the statutory language is determinative.
[Best Practice for Voting Officials] As mandated by HAVA
301(a)(1)(C), the voting system notifies the voter of an attempted
overvote in a way that preserves the privacy of the voter and the
confidentiality of the ballot.

2. Voter Anonymity Shall Be Maintained for Alternative Format Ballot
Presentation

2.1 No Information Shall Be Kept Within a Non-Paper-Based Cast Vote
Record That Identifies Any Accessibility Feature(s) Used by a Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Large-print paper ballots unavoidably preserve such
information.
2.1.1 No Information Shall Be Kept Within a Non-Paper-Based Cast Vote
Record That Identifies Any Alternative Language Feature(s) Used by a
Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Non-English paper ballots unavoidably preserve such
information.
[Best Practice for Voting Officials] Appropriate procedures are
needed to ensure that absentee balloting enable the voter to preserve
privacy. There is no practical means to prevent a voter from revealing
an absentee paper ballot to others. But the procedures should ensure
that if a voter chooses to maintain privacy, it is not violated at a
later stage, in particular when the ballot is received by voting
officials.
2.2.8 Vote Tabulating Program
Each voting system shall have a vote tabulation program that will
meet specific functional requirements.
2.2.8.1 Functions
The vote tabulating program software resident in each voting
device, vote count server, or other devices shall include all software
modules required to:
a. Monitor system status and generate machine-level audit reports;
b. Accommodate device control functions performed by polling place
officials and maintenance personnel;
c. Register and accumulate votes; and
d. Accommodate variations in ballot counting logic.
2.2.8.2 Voting Variations
There are significant variations among the election laws of the 50
states with respect to permissible ballot contents, voting options, and
the associated ballot counting logic. The TDP accompanying the system
shall specifically identify which of the following items can and cannot
be supported by the system, as well as how the system can implement the
items supported:
a. Closed primaries;
b. Open primaries;
c. Partisan offices;
d. Non-partisan offices;
e. Write-in voting;
f. Primary presidential delegation nominations;
g. Ballot rotation;
h. Straight party voting;
i. Cross-party endorsement;
j. Split precincts;
k. Vote for N of M;
l. Recall issues, with options;
m. Cumulative voting;
n. Ranked order voting; and
o. Provisional or challenged ballots.
2.2.9 Ballot Counter
For all voting systems, each device that tabulates ballots shall
provide a counter that:
a. Can be set to zero before any ballots are submitted for tally;
b. Records the number of ballots cast during a particular test
cycle or election;
c. Increases the count only by the input of a ballot;
d. Prevents or disables the resetting of the counter by any person
other than authorized persons at authorized points; and
e. Is visible to designated election officials.
2.2.10 Telecommunications
For all voting systems that use telecommunications for the
transmission of data during pre-voting, voting or post-voting
activities, capabilities shall be provided that ensure data are
transmitted with no alteration or unauthorized disclosure during
transmission. Such transmissions shall not violate the privacy,
secrecy, and integrity demands of the Standards. Section 5 of the
Standards describes telecommunications standards that apply to, at a
minimum, the following types of data transmissions:
Voter Authentication: Coded information that confirms the
identity of a voter for security purposes for a system that transmit
votes individually over a public network;
Ballot Definition: Information that describes to a voting
machine the content and appearance of the ballots to be used in an
election;
Vote Transmission to Central Site: For systems that
transmit votes individually over a public network, the transmission of
a single vote to the county (or contractor) for consolidation with
other county vote data;
Vote Count: Information representing the tabulation of
votes at any one of several levels: polling place, precinct, or central
count; and
List of Voters: A listing of the individual voters who
have cast ballots in a specific election.
2.2.9 Data Retention
United States Code Title 42, Sections 1974 through 1974e, states
that election administrators shall preserve for 22 months ``all records
and paper that came into (their) possession relating to an application,
registration, payment of poll tax, or other act requisite to voting.''
This retention requirement applies to systems that will be used at
anytime for voting of candidates for Federal offices (e.g., Member of
Congress, United States Senator, and/or Presidential Elector).
Therefore, all systems shall provide for maintaining the integrity of
voting and audit data during an election and for a period of at least
22 months thereafter.
Because the purpose of this law is to assist the Federal government
in discharging its law enforcement responsibilities in connection with
civil rights and elections crimes, its scope must be interpreted in
keeping with that objective. The appropriate state or local authority
must preserve all records that may be relevant to the detection and
prosecution of federal civil rights or election crimes for the 22-month
federal retention period, if the records were generated in connection
with an election that was held in whole or in part to select federal
candidates. It is important to note that Section 1974 does not require
that election officials generate any specific type or classification of
election record. However, if a record is generated, Section 1974 comes
into force and the appropriate authority must retain the records for 22
months.
For 22-month document retention, the general rule is that all
printed copy

[[Page 18955]]

records produced by the election database and ballot processing systems
shall be so labeled and archived. Regardless of system type, all audit
trail information spelled out in subsection 4.5 of the Standards shall
be retained in its original format, whether that be real-time logs
generated by the system, or manual logs maintained by election
personnel. The election audit trail includes not only in-process logs
of election-night (and subsequent processing of absentee or provisional
ballots), but also time logs of baseline ballot definition formats, and
system readiness and testing results.
In many voting systems, the source of election-specific data (and
ballot formats) is a database or file. In precinct count systems, this
data is used to program each machine, establish ballot layout, and
generate tallying files. It is not necessary to retain this information
on electronic media if there is an official, authenticatable printed
copy of all final database information. However, it is recommended that
the state or local jurisdiction also retain electronic records of the
aggregate data for each device so that reconstruction of an election is
possible without data re-entry. The same requirement and recommendation
applies to vote results generated by each precinct device or system.

2.3 Pre-Voting Functions

This section defines capabilities required to support functions
performed prior to the opening of polls. All voting systems shall
provide capabilities to support:
Ballot preparation;
Election programming;
Ballot and program installation and control;
Readiness testing;
Verification at the polling place; and
Verification at the central counting place.
The standards also include requirements to ensure compatible
interfaces with the ballot definition process and the reporting of
election results.
2.3.1 Ballot Preparation
Ballot preparation is the process of using election databases to
define the specific contests, questions, and related instructions to be
contained in ballots and to produce all permissible ballot layouts.
Ballot preparation requirements include:
General capabilities for ballot preparation;
Ballot formatting; and
Ballot production.
2.3.1.1 General Capabilities
All systems shall provide the general capabilities for ballot
preparation.
2.3.1.1.1 Common Standards
All systems shall be capable of:
a. Enabling the automatic formatting of ballots in accordance with
the requirements for offices, candidates, and measures qualified to be
placed on the ballot for each political subdivision and election
district;
b. Collecting and maintaining the following data:
(1) Offices and their associated labels and instructions;
(2) Candidate names and their associated labels; and
(3) Issues or measures and their associated text;
c. Supporting the maximum number of potentially active voting
positions as indicated in the system documentation;
d. For a primary election, generating ballots that segregate the
choices in partisan races by party affiliation;
e. Generating ballots that contain identifying codes or marks
uniquely associated with each format; and
f. Ensuring that vote response fields, selection buttons, or
switches properly align with the specific candidate names and/or issues
printed on the ballot display, ballot card or sheet, or separate ballot
pages.
2.3.1.1.2 Paper-Based System Standards
In addition to the common standards, paper-based systems shall meet
the following standards applicable to the technology used:
a. Enable voters to make selections by punching a hole or by making
a mark in areas designated for this purpose upon each ballot card or
sheet;
b. For punchcard systems, ensure that the vote response fields can
be properly aligned with punching devices used to record votes; and
c. For marksense systems, ensure that the timing marks align
properly with the vote response fields.
2.3.1.2 Ballot Formatting
Ballot formatting is the process by which election officials or
their designees use election databases and vendor system software to
define the specific contests and related instructions contained on the
ballot and present them in a layout permitted by state law. All systems
shall provide a capability for:
a. Creation of newly defined elections;
b. Rapid and error-free definition of elections and their
associated ballot layouts;
c. Uniform allocation of space and fonts used for each office,
candidate, and contest such that the voter perceives no active voting
position to be preferred to any other;
d. Simultaneous display of the maximum number of choices for a
single contest as indicated by the vendor in the system documentation;
e. Retention of previously defined formats for an election;
f. Prevention of unauthorized modification of any ballot formats;
and
g. Modification by authorized persons of a previously defined
ballot format for use in a subsequent election.
2.3.1.3 Ballot Production
Ballot production is the process of converting ballot formats to a
media ready for use in the physical ballot production or electronic
presentation.
2.3.1.3.1 Common Standards
The voting system shall provide a means of printing or otherwise
generating a ballot display that can be installed in all system voting
devices for which it is intended. All systems shall provide a
capability to ensure:
a. The electronic display or printed document on which the user
views the ballot is capable of rendering an image of the ballot in any
of the languages required by The Voting Rights Act of 1965, as amended;
b. The electronic display or printed document on which the user
views the ballot does not show any advertising or commercial logos of
any kind, whether public service, commercial, or political, unless
specifically provided for in State law. Electronic displays shall not
provide connection to such material through hyperlink; and
c. The ballot conforms to vendor specifications for type of paper
stock, weight, size, shape, size and location of punch or mark field
used to record votes, folding, bleed through, and ink for printing if
paper ballot documents or paper displays are part of the system.
2.3.1.3.2 Paper-Based System Standards
In addition to the common standards, vendor documentation for
marksense systems shall include specifications for ballot materials to
ensure that vote selections are read from only a single ballot at a
time, without detection of marks from multiple ballots concurrently
(e.g., reading of bleed-through from other ballots).
2.3.2 Election Programming
Election programming is the process by which election officials or
their designees use election databases and vendor system software to
logically define the voter choices associated with

[[Page 18956]]

the contents of the ballots. All systems shall provide for the:
a. Logical definition of the ballot, including the definition of
the number of allowable choices for each office and contest;
b. Logical definition of political and administrative subdivisions,
where the list of candidates or contests varies between polling places;
c. Exclusion of any contest on the ballot in which the voter is
prohibited from casting a ballot because of place of residence, or
other such administrative or geographical criteria;
d. Ability to select from a range of voting options to conform to
the laws of the jurisdiction in which the system will be used; and
e. Generation of all required master and distributed copies of the
voting program, in conformance with the definition of the ballots for
each voting device and polling place, and for each tabulating device.
2.3.3 Ballot and Program Installation and Control
All systems shall provide a means of installing ballots and
programs on each piece of polling place or central count equipment in
accordance with the ballot requirements of the election and the
requirements of the jurisdiction in which the equipment will be used.
All systems shall include the following at the time of ballot and
program installation:
a. A detailed work plan or other documentation providing a schedule
and steps for the software and ballot installation, which includes a
table outlining the key dates, events and deliverables;
b. A capability for automatically verifying that the software has
been properly selected and installed in the equipment or in a
programmable memory devices and for indicating errors; and
c. A capability for automatically validating that software
correctly matches the ballot formats that it is intended to process,
for detecting errors, and for immediately notifying an election
official of detected errors.
2.3.4 Readiness Testing
Election personnel conduct equipment and system readiness tests
prior to the start of an election to ensure that the voting system
functions properly, to confirm that system equipment has been properly
integrated, and to obtain equipment status reports.
2.3.4.1 Common Standards
All systems shall provide the capabilities to:
a. Verify that voting machines or vote recording and data
processing equipment, precinct count equipment, and central count
equipment are properly prepared for an election, and collect data that
verifies equipment readiness;
b. Obtain status and data reports from each set of equipment;
c. Verify the correct installation and interface of all system
equipment;
d. Verify that hardware and software function correctly;
e. Generate consolidated data reports at the polling place and
higher jurisdictional levels; and
f. Segregating test data from actual voting data, either
procedurally or by hardware/software features.
Resident test software, external devices, and special purpose test
software connected to or installed in voting devices to simulate
operator and voter functions may be used for these tests provided that
the following standards are met:
a. These elements shall be capable of being tested separately, and
shall be proven to be reliable verification tools prior to their use;
and
b. These elements shall be incapable of altering or introducing any
residual effect on the intended operation of the voting device during
any succeeding test and operational phase.
2.3.4.2 Paper-Based Systems
Paper-based systems shall:
a. Support conversion testing that uses all potential ballot
positions as active positions; and
b. Support conversion testing of ballots with active position
density for systems without pre-designated ballot positions.
2.3.5 Verification at the Polling Place
Election officials perform verification at the polling place to
ensure that all voting systems and equipment function properly before
and during an election. All systems shall provide a formal record of
the following, in any media, upon verification of the authenticity of
the command source:
a. The election's identification data;
b. The identification of all equipment units;
c. The identification of the polling place;
d. The identification of all ballot formats;
e. The contents of each active candidate register by office and of
each active measure register at all storage locations (showing that
they contain only zeros);
f. A list of all ballot fields that can be used to invoke special
voting options; and
g. Other information needed to confirm the readiness of the
equipment, and to accommodate administrative reporting requirements.
To prepare voting devices to accept voted ballots, all voting
systems shall provide the capability to test each device prior to
opening to verify that each is operating correctly. At a minimum, the
tests shall include:
a. Confirmation that there are no hardware or software failures;
and
b. Confirm that the device is ready to be activated for accepting
votes.
If a precinct count system includes equipment for the consolidation
of polling place data at one or more central counting places, it shall
have means to verify the correct extraction of voting data from
transportable memory devices, or to verify the transmission of secure
data over secure communication links.
2.3.6 Verification at the Central Location
Election officials perform verification at the central location to
ensure that vote counting and vote consolidation equipment and software
function properly before and after an election. Upon verification of
the authenticity of the command source, any system used in a central
count environment shall provide a printed record of the following :
a. The election's identification data;
b. The contents of each active candidate register by office and of
each active measure register at all storage locations (showing that
they contain all zeros); and
c. Other information needed to ensure the readiness of the
equipment and to accommodate administrative reporting requirements.

2.4 Voting Functions

All systems shall support:
[sdiam] Opening the polls; and
[sdiam] Casting a ballot.
Additionally, all DRE systems shall support:
[sdiam] Activating the ballot.
[sdiam] Augmenting the election counter; and
[sdiam] Augmenting the life-cycle counter.
2.4.1 Opening the Polls
The capabilities required for opening the polls are specific to
individual voting system technologies. At a minimum, the systems shall
provide the functional capabilities indicated below.
2.4.1.1 Opening the Polling Place (Precinct Count Systems)
To allow voting devices to be activated for voting, the system
shall provide:

[[Page 18957]]

a. An internal test or diagnostic capability to verify that all of
the polling place tests specified in Section 2.3.5 have been
successfully completed; and
b. Automatic disabling any device that has not been tested until it
has been tested.
2.4.1.2 Paper-Based System Standards
The standards for opening the polling place for paper-based systems
consist of common standards and additional standards that apply to
precinct count paper-based systems.
2.4.1.2.1 All Paper-Based Systems
To facilitate opening the polls, all paper-based systems shall
include:
a. A means of verifying that ballot punching or marking devices are
properly prepared and ready to use;
b. A voting booth or similar facility, in which the voter may punch
or mark the ballot in privacy; and
c. Secure receptacles for holding voted ballots.
2.4.1.2.2 Precinct Count Paper-Based Systems
In addition to the above requirements, all paper-based precinct
count equipment shall include a means of:
a. Activating the ballot counting device;
b. Verifying that the device has been correctly activated and is
functioning properly; and
c. Identifying device failure and corrective action needed.
2.4.1.3 DRE System Standards
To facilitate opening the polls, all DRE systems shall include:
a. A security seal, a password, or a data code recognition
capability to prevent the inadvertent or unauthorized actuation of the
poll-opening function;
b. A means of enforcing the execution of steps in the proper
sequence if more than one step is required;
c. A means of verifying the system has been activated correctly;
and
d. A means of identifying system failure and any corrective action
needed.
2.4.2 Activating the Ballot (DRE Systems)
To activate the ballot, all DRE systems shall:
a. Enable election officials to control the content of the ballot
presented to the voter, whether presented in printed form or electronic
display, such that each voter is permitted to record votes only in
contests in which that voter is authorized to vote;
b. Allow each eligible voter to cast a ballot;
c. Prevent a voter from voting on a ballot to which he or she is
not entitled; and
d. Prevent a voter from casting more than one ballot in the same
election.
e. Activate the casting of a ballot in a general election;
f. Enable the selection of the ballot that is appropriate to the
party affiliation declared by the voter in a primary election;
g. Activate all portions of the ballot upon which the voter is
entitled to vote; and
h. Disable all portions of the ballot upon which the voter is not
entitled to vote.
2.4.3 Casting a Ballot
Some required capabilities for casting a ballot are common to all
systems. Others are specific to individual voting technologies or
intended use. Systems must provide additional functional capabilities
that enable accessibility to disabled voters as defined in Section
2.2.7 of the Standards.
2.4.3.1 Common Standards
To facilitate casting a ballot, all systems shall:
a. Provide text that is at least 3 millimeters high and provide the
capability to adjust or magnify the text to an apparent size of 6.3
millimeters;
b. Protect the secrecy of the vote such that the system cannot
reveal any information about how a particular voter voted, except as
otherwise required by individual State law;
c. Record the selection and non-selection of individual vote
choices for each contest and ballot measure;
d. Record the voter's selection of candidates whose names do not
appear on the ballot, if permitted under State law, and record as many
write-in votes as the number of candidates the voter is allowed to
select;
e. In the event of a failure of the main power supply external to
the voting system, provide the capability for any voter who is voting
at the time to complete casting a ballot, allow for the graceful
shutdown of the voting system without loss or degradation of the voting
and audit data, and allow voters to resume voting once the voting
system has reverted to back-up power; and
f. Provide the capability for voters to continue casting ballots in
the event of a failure of a telecommunications connection within the
polling place or between the polling place and any other location.
2.4.3.2 Paper-Based Systems Standards
The standards for casting a ballot for paper-based systems consist
of common standards and additional standards that apply to precinct
count paper-based systems.
2.4.3.2.1 All Paper-Based Systems
All paper-based systems shall:
a. Allow the voter to easily identify the voting field that is
associated with each candidate or ballot measure response;
b. Allow the voter to punch or mark the ballot to register a vote;
c. Allow either the voter or the appropriate election official to
place the voted ballot into the ballot counting device (for precinct
count systems) or into a secure receptacle (for central count systems);
and
d. Protect the secrecy of the vote throughout the process.
2.4.3.2.2 Precinct Count Paper-Based Systems
In addition to the above requirements, all paper-based precinct
count systems shall:
a. Provide feedback to the voter that identifies specific contests
or ballot issues for which an overvote or undervote is detected;
b. Allow the voter, at the voter's choice, to vote a new ballot or
submit the ballot `as is' without correction; and
c. Allow an authorized election official to turn off the
capabilities defined in `a' and `b' above.
2.4.3.3 DRE Systems Standards
In addition to the above common requirements, DRE systems shall:
a. Prohibit the voter from accessing or viewing any information on
the display screen that has not been authorized by election officials
and preprogrammed into the voting system (i.e., no potential for
display of external information or linking to other information
sources);
b. Enable the voter to easily identify the selection button or
switch, or the active area of the ballot display that is associated
with each candidate or ballot measure response;
c. Allow the voter to select his or her preferences on the ballot
in any legal number and combination;
d. Indicate that a selection has been made or canceled;
e. Indicate to the voter when no selection, or an insufficient
number of selections, has been made in a contest;
f. Prevent the voter from overvoting;
g. Notify the voter when the selection of candidates and measures
is completed;
h. Allow the voter, before the ballot is cast, to review his or her
choices and, if the voter desires, to delete or change his or her
choices before the ballot is cast;

[[Page 18958]]

i. For electronic image displays, prompt the voter to confirm the
voter's choices before casting his or her ballot, signifying to the
voter that casting the ballot is irrevocable and directing the voter to
confirm the voter's intention to cast the ballot;
j. Notify the voter after the vote has been stored successfully
that the ballot has been cast;
k. Notify the voter that the ballot has not been cast successfully
if it is not stored successfully, including storage of the ballot
image, and provide clear instruction as to the steps the voter should
take to cast his or her ballot should this event occur;
l. Provide sufficient computational performance to provide
responses back to each voter entry in no more than three seconds;
m. Ensure that the votes stored accurately represent the actual
votes cast;
n. Prevent modification of the voter's vote after the ballot is
cast;
o. Provide a capability to retrieve ballot images in a form
readable by humans (in accordance with the requirements of Section
2.2.2.2 and 2.2.4.2);
p. Increment the proper ballot position registers or counters;
q. Protect the secrecy of the vote throughout the voting process;
r. Prohibit access to voted ballots until after the close of polls;
s. Provide the ability for election officials to submit test
ballots for use in verifying the end-to-end integrity of the system;
and
t. Isolate test ballots such that they are accounted for accurately
in vote counts and are not reflect in official vote counts for specific
candidates or measures.

2.5 Post-Voting Functions

All systems shall provide capabilities to accumulate and report
results for the jurisdiction and to generate audit trails. In addition,
precinct count systems must provide a means to close the polling place
including generating appropriate reports. If the system provides the
capability to broadcast results, additional standards apply.
2.5.1 Closing the Polling Place (Precinct Count)
These standards for closing the polling place are specific to
precinct count systems. The system shall provide the means for:
a. Preventing the further casting of ballots once the polling place
has closed;
b. Providing an internal test that verifies that the prescribed
closing procedure has been followed, and that the device status is
normal;
c. Incorporating a visible indication of system status;
d. Producing a diagnostic test record that verifies the sequence of
events, and indicates that the extraction of voting data has been
activated; and
e. Precluding the unauthorized reopening of the polls once the poll
closing has been completed for that election.
2.5.2 Consolidating Vote Data
All systems shall provide a means to consolidate vote data from all
polling places, and optionally from other sources such as absentee
ballots, provisional ballots, and voted ballots requiring human review
(e.g., write-in votes).
2.5.3 Producing Reports
All systems shall be able to create reports summarizing the data on
multiple levels.
2.5.3.1 Common Standards
All systems shall provide capabilities to:
a. Support geographic reporting, which requires the reporting of
all results for each contest at the precinct level and additional
jurisdictional levels;
b. Produce a printed report of the number of ballots counted by
each tabulator;
c. Produce a printed report for each tabulator of the results of
each contest that includes the votes cast for each selection, the count
of undervotes, and the count of overvotes;
d. Produce a consolidated printed report of the results for each
contest of all votes cast (including the count of ballots from other
sources supported by the system as specified by the vendor) that
includes the votes cast for each selection, the count of undervotes,
and the count of overvotes;
e. Be capable of producing a consolidated printed report of the
combination of overvotes for any contest that is selected by an
authorized official (e.g.; the number of overvotes in a given contest
combining candidate A and candidate B, combining candidate A and
candidate C, etc.);
f. Produce all system audit information required in Section 4.5 in
the form of printed reports, or in electronic memory for printing
centrally; and
g. Prevent data from being altered or destroyed by report
generation, or by the transmission of results over telecommunications
lines.
2.5.3.2 Precinct Count Systems
In addition to the common reporting requirements, all precinct
count voting systems shall:
a. Prevent the printing of reports and the unauthorized extraction
of data prior to the official close of the polling place;
b. Provide a means to extract information from a transportable
programmable memory device or data storage medium for vote
consolidation;
c. Consolidate the data contained in each unit into a single report
for the polling place when more than one voting machine or precinct
tabulator is used; and
d. Prevent data in transportable memory from being altered or
destroyed by report generation, or by the transmission of results over
telecommunications lines.
2.5.4 Broadcasting Results
Some voting systems offer the capability to make unofficial results
available to external organizations such as the news media, political
party officials, and others. Although this capability is not required,
systems that make unofficial results available shall:
a. Provide only aggregated results, and not data from individual
ballots;
b. Provide no access path from unofficial electronic reports or
files to the storage devices for official data; and
c. Clearly indicate on each report or file that the results it
contains are unofficial.
2.6 Maintenance, Transportation, and Storage
All systems shall be designed and manufactured to facilitate
preventive and corrective maintenance, conforming to the hardware
standards described in Section 3.
All vote casting and tally equipment designated for storage between
elections shall:
a. Function without degradation in capabilities after transit to
and from the place of use, as demonstrated by meeting the performance
standards described in Section 3; and
b. Function without degradation in capabilities after storage
between elections, as demonstrated by meeting the performance standards
described in Section 3.

Volume I, Section 3

Table of Contents

3 Hardware Standards

3.1 Scope
3.1.1 Hardware Sources
3.1.2 Organization of this Section
3.2 Performance Requirements
3.2.1 Accuracy Requirements

[[Page 18959]]

3.2.2 Environmental Requirements
3.2.2.1 Shelter Requirements
3.2.2.2 Space Requirements
3.2.2.3 Furnishings and Fixtures
3.2.2.4 Electrical Supply
3.2.2.5 Electrical Power Disturbance
3.2.2.6 Electrical Fast Transient
3.2.2.7 Lightning Surge
3.2.2.8 Electrostatic Disruption
3.2.2.9 Electromagnetic Radiation
3.2.2.10 Electromagnetic Susceptibility
3.2.2.11 Conducted RF Immunity
3.2.2.12 Magnetic Fields Immunity
3.2.2.13 Environmental Control--Operating Environment
3.2.2.14 Environmental Control--Transit and Storage
3.2.2.15 Data Network Requirements
3.2.3 Election Management System (EMS) Requirements
3.2.3.1 Recording Requirements
3.2.3.2 Memory Stability
3.2.4 Vote Recording Requirements
3.2.4.1 Common Standards
3.2.4.2 Paper-Based Recording Standards
3.2.4.2.1 Paper Ballot Standards
3.2.4.2.2 Punching Devices
3.2.4.2.3 Marking Devices
3.2.4.2.4 Frames or Fixtures for Punchcard Ballots
3.2.4.2.5 Frames or Fixtures for Printed Ballots
3.2.4.2.6 Ballot Boxes and Ballot Transfer Boxes
3.2.4.3 DRE Systems Recording Requirements
3.2.4.3.1 Activity Indicator
3.2.4.3.2 DRE System Vote Recording
3.2.4.3.3 Recording Accuracy
3.2.4.3.4 Recording Reliability
3.2.5 Paper-based Conversion Requirements
3.2.5.1 Ballot Handling
3.2.5.1.1 Capacity (Central Count)
3.2.5.1.2 Exception Handling (Central Count)
3.2.5.1.3 Exception Handling (Precinct Count)
3.2.5.1.4 Multiple Feed Prevention
3.2.5.2 Ballot Reading Accuracy
3.2.6 Processing Requirements
3.2.6.1 Paper-Based System Processing Requirements
3.2.6.1.1 Processing Accuracy
3.2.6.1.2 Memory Stability
3.2.6.2 DRE System Processing Requirements
3.2.6.2.1 Processing Speed
3.2.6.2.2 Processing Accuracy
3.2.6.2.3 Memory Stability
3.2.7 Reporting Requirements
3.2.7.1 Removable Storage Media
3.2.7.2 Printers
3.2.8 Vote Data Management Requirements
3.2.8.1 Data File Management
3.2.8.2 Data Report Generation
3.3 Physical Characteristics
3.3.1 Size
3.3.2 Weight
3.3.3 Transport and Storage of Precinct Systems
3.4 Design, Construction, and Maintenance Characteristics
3.4.1 Materials, Processes, and Parts
3.4.2 Durability
3.4.3 Reliability
3.4.4 Maintainability
3.4.4.1 Physical Attributes
3.4.4.2 Additional Attributes
3.4.5 Availability
3.4.6 Product Marking
3.4.7 Workmanship
3.4.8 Safety

3 Hardware Standards

3.1 Scope

This section contains the requirements for the machines and
manufactured devices that are part of a voting system. It specifies
minimum values for certain performance characteristics; physical
characteristics; and design, construction, and maintenance
characteristics for the hardware and selected related components of all
voting systems, such as:
Ballot printers;
Ballot cards and sheets;
Ballot displays;
Voting devices, including punching and marking devices and
DRE recording devices;
Voting booths and enclosures;
Ballot boxes and ballot transfer boxes;
Ballot readers;
Computers used to prepare ballots, program elections,
consolidate and report votes, and perform other elections management
activities;
Electronic ballot recorders;
Electronic precinct vote control units;
Removable electronic data storage media;
Servers; and
Printers.
This section applies to the combination of software and hardware to
accomplish specific performance and system control requirements.
Standards that are specific to software alone are provided in Section 4
of the Standards.
3.1.1 Hardware Sources
The requirements of this section apply generally to all hardware
used in voting systems, including:
a. Hardware provided by the voting system vendor and its suppliers;
b. Hardware furnished by an external provider (for example,
providers of commercial off-the-shelf (COTS) machines and devices)
where the hardware may be used in any way during voting system
operation; and
c. Hardware provided by the voting jurisdiction.
3.1.2 Organization of this Section
The standards presented in this section are organized as follows:
Performance Requirements: These requirements address the
combined operational capabilities of the voting system's hardware and
software across a broad range of parameters;
Physical Requirements: These requirements address the
size, weight and transportability of the voting system; and
Design, Construction, and Maintenance Requirements: These
requirements address the reliability and durability of materials,
product marking, quality of system workmanship, safety, and other
attributes to ensure smooth system operation in the voting environment.
3.2 Performance Requirements
The performance requirements address a broad range of parameters,
encompassing:
a. Accuracy requirements, where requirements are specified for
distinct processing functions of paper-based and DRE systems;
b. Environmental requirements, where no distinction is made between
requirements for paper-based and DRE systems, but requirements for
precinct and central count are described;
c. Vote data management requirements, where no differentiation is
made between requirements for paper-based and DRE systems;
d. Vote recording requirements, where separate and distinct
requirements are delineated for paper-based and DRE systems;
e. Conversion requirements, which apply only to paper-based
systems;
f. Processing requirements, where separate and distinct
requirements are delineated for paper-based and DRE systems; and
g. Reporting requirements, where no distinction is made between
requirements for paper-based and DRE systems, but where differences
between precinct and central count systems are readily apparent based
on differences of their reporting.
The performance requirements include such attributes as ballot
reading and handling requirements; system accuracy; memory stability;
and the ability to withstand specified environmental conditions. These
characteristics also encompass system-wide requirements for shelter,
electrical supply, and compatibility with data networks.
Performance requirements for voting systems represent the combined
operational capability of both system hardware and software. Accuracy,
as measured by data error rate, and operational failure are treated as
distinct attributes in performance testing. All systems shall meet the
performance requirements under operating

[[Page 18960]]

conditions and after storage under non-operating conditions.
3.2.1 Accuracy Requirements
Voting system accuracy addresses the accuracy of data for each of
the individual ballot positions that could be selected by a voter,
including the positions that are not selected. For a voting system,
accuracy is defined as the ability of the system to capture, record,
store, consolidate and report the specific selections and absence of
selections, made by the voter for each ballot position without error.
Required accuracy is defined in terms of an error rate that for testing
purposes represents the maximum number of errors allowed while
processing a specified volume of data. This rate is set at a
sufficiently stringent level such that the likelihood of voting system
errors affecting the outcome of an election is exceptionally remote
even in the closest of elections.
The error rate is defined using a convention that recognizes
differences in how vote data is processed by different types of voting
systems. Paper-based and DRE systems have different processing steps.
Some differences also exist between precinct count and central count
systems. Therefore, the acceptable error rate applies separately and
distinctly to each of the following functions:
a. For all paper-based systems:
(1) Scanning ballot positions on paper ballots to detect selections
for individual candidates and contests;
(2) Conversion of selections detected on paper ballots into digital
data;
b. For all DRE systems:
(1) Recording the voter selections of candidates and contests into
voting data storage; and
(2) Independently from voting data storage, recording voter
selections of candidates and contests into ballot image storage.
c. For precinct-count systems (paper-based and DRE):
Consolidation of vote selection data from multiple precinct-based
systems to generate jurisdiction-wide vote counts, including storage
and reporting of the consolidated vote data; and
d. For central-count systems (paper-based and DRE):
Consolidation of vote selection data from multiple counting devices
to generate jurisdiction-wide vote counts, including storage and
reporting of the consolidated vote data.
For testing purposes, the acceptable error rate is defined using
two parameters: The desired error rate to be achieved, and the maximum
error rate that should be accepted by the test process.
For each processing function indicated above, the system shall
achieve a target error rate of no more than one in 10,000,000 ballot
positions, with a maximum acceptable error rate in the test process of
one in 500,000 ballot positions.
3.2.2 Environmental Requirements
The environmental requirements for voting systems include shelter,
space, furnishings and fixtures, supplied energy, environmental control
, and external telecommunications services. Environmental conditions
applicable to the design and operation of voting systems consist of the
following categories:
Natural environment, including temperature, humidity, and
atmospheric pressure;
Induced environment, including proper and improper
operation and handling of the system and its components during the
election processes;
Transportation and storage; and
Electromagnetic signal environment, including exposure to
and generation of radio frequency energy.
All voting systems shall be designed to withstand the environmental
conditions contained in the appropriate test procedures of the
Standards. These procedures will be applied to all devices for casting,
scanning and counting ballots, except those that constitute COTS
devices that have not been modified in any manner to support their use
as part of a voting system and that have a documented record of
performance under conditions defined in the Standards.
The TDP supplied by the vendor shall include a statement of all
requirements and restrictions regarding environmental protection,
electrical service, recommended auxiliary power, telecommunications
service, and any other facility or resource required for the proper
installation and operation of the system.
3.2.2.1 Shelter Requirements
All precinct count systems shall be designed for storage and
operation in any enclosed facility ordinarily used as a warehouse or
polling place, with prominent instructions as to any special storage
requirements.
3.2.2.2 Space Requirements
There is no restriction on space allowed for the installation of
voting systems, except that the arrangement of these systems shall not
impede performance of their duties by polling place officials, the
orderly flow of voters through the polling place, or the ability for
the voter to vote in private.
3.2.2.3 Furnishings and Fixtures
Any furnishings or fixtures provided as a part of voting systems,
and any components provided by the vendor that are not a part of the
system but that are used to support its storage, transportation, or
operation, shall comply with the design and safety requirements of
Subsection 3.4.8.
3.2.2.4 Electrical Supply
Components of voting systems that require an electrical supply
shall meet the following standards:
a. Precinct count systems shall operate with the electrical supply
ordinarily found in polling places (120vac/60hz/1);
b. Central count systems shall operate with the electrical supply
ordinarily found in central tabulation facilities or computer room
facilities (120vac/60hz/1, 208vac/60hz/3, or 240vac/60hz/2); and
c. All systems shall also be capable of operating for a period of
at least 2 hours on backup power, such that no voting data is lost or
corrupted, nor normal operations interrupted. When backup power is
exhausted the system shall retain the contents of all memories intact.
The backup power capability is not required to provide lighting of
the voting area.
3.2.2.5 Electrical Power Disturbance
Vote scanning and counting equipment for paper-based systems, and
all DRE equipment, shall be able to withstand, without disruption of
normal operation or loss of data:
d. Surges of 15% line variations of nominal line
voltage; and
e. Electric power increases of 7.5% and reductions of 12.5% of
nominal specified power supply for a period of up to four hours at each
power level.
3.2.2.6 Electrical Fast Transient
Vote scanning and counting equipment for paper-based systems, and
all DRE equipment, shall be able to withstand, without disruption of
normal operation or loss of data, electrical fast transients of:
a. 2 kV AC & DC external power lines;
b. 1 kV all external wires >3m no control; and
c. 2 kV all external wires control.
3.2.2.7 Lightning Surge
Vote scanning and counting equipment for paper-based systems, and

[[Page 18961]]

all DRE equipment, shall be able to withstand, without disruption of
normal operation or loss of data, surges of:
a. 2 kV AC line to line;
b. 2 kV AC line to earth;
c. .5 kV DC line to line >10m;
d. .5 kV DC line to earth >10m; and
e. 1 kV I/O sig/control >30m.
3.2.2.8 Electrostatic Disruption
Vote scanning and counting equipment for paper-based systems, and
all DRE equipment, shall be able to withstand 15 kV air
discharge and 8 kV contact discharge without damage or loss
of data. The equipment may reset or have momentary interruption so long
as normal operation is resumed without human intervention or loss of
data. Loss of data means votes that have been completed and confirmed
to the voter.
3.2.2.9 Electromagnetic Radiation
Vote scanning and counting equipment for paper-based systems, and
all DRE equipment, shall comply with the Rules and Regulations of the
Federal Communications Commission, Part 15, Class B requirements for
both radiated and conducted emissions.
3.2.2.10 Electromagnetic Susceptibility
Vote scanning and counting equipment for paper-based systems, and
all DRE equipment, shall be able to withstand an electromagnetic field
of 10 V/m modulated by a 1 kHz 80% AM modulation over the frequency
range of 80 MHz to 1000 MHz, without disruption of normal operation or
loss of data.
3.2.2.11 Conducted RF Immunity
Vote scanning and counting equipment for paper-based systems, and
all DRE equipment, shall be able to withstand, without disruption of
normal operation or loss of data, conducted RF energy of:
a. 10V AC & DC power; and
b. 10V, 20 sig/control >3m.
3.2.2.12 Magnetic Fields Immunity
Vote scanning and counting equipment for paper-based systems, and
all DRE equipment, shall be able to withstand, without disruption of
normal operation or loss of data, AC magnetic fields of 30 A/m at 60
Hz.
3.2.2.13 Environmental Control--Operating Environment
Equipment used for election management activities or vote counting
(including both precinct and central count systems) shall be capable of
operation in temperatures ranging from 50 to 95 degrees Fahrenheit.
3.2.2.14 Environmental Control--Transit and Storage
Equipment used for vote casting, or for counting votes in a
precinct count system, shall meet specific minimum performance
standards that simulate exposure to physical shock and vibration
associated with handling and transportation by surface and air common
carriers, and to temperature conditions associated with delivery and
storage in an uncontrolled warehouse environment.
a. High and low storage temperatures ranging from -4 to +140
degrees Fahrenheit, equivalent to MIL-STD-810D, Methods 501.2 and
502.2, Procedure I-Storage;
b. Bench handling equivalent to the procedure of MIL-STD-810D,
Method 516.3, Procedure VI;
c. Vibration equivalent to the procedure of MIL-STD-810D, Method
514.3, Category 1--Basic Transportation, Common Carrier; and
d. Uncontrolled humidity equivalent to the procedure of MIL-STD-
810D, Method 507.2, Procedure I--Natural Hot--Humid.
3.2.2.15 Data Network Requirements
Voting systems may use a local or remote data network. If such a
network is used, then all components of the network shall comply with
the telecommunications requirements described in Section 5 of the
Standards and the Security requirements described in Section 6.
3.2.3 Election Management System (EMS) Requirements
The EMS requirements address electronic hardware and software used
to conduct the pre-voting functions defined in Section 2 with regard to
ballot preparation, election programming, ballot and program
installation, readiness testing, verification at the polling place, and
verification at the central location.
3.2.3.1 Recording Requirements
Voting systems shall accurately record all election management data
entered by the user, including election officials or their designees.
For recording accuracy, all systems shall:
a. Record every entry made by the user;
b. Add permissible voter selections correctly to the memory
components of the device;
c. Verify the correctness of detection of the user selections and
the addition of the selections correctly to memory;
d. Add various forms of data entered directly by the election
official or designee, such as text, line art, logos, and images;
e. Verify the correctness of detection of data entered directly by
the user and the addition of the selections correctly to memory;
f. Preserve the integrity of election management data stored in
memory against corruption by stray electromagnetic emissions, and
internally generated spurious electrical signals; and
g. Log corrected data errors by the system.
3.2.3.2 Memory Stability
Electronic system memory devices, used to retain election
management data, shall have demonstrated error-free data retention for
a period of 22 months.
3.2.4 Vote Recording Requirements
The vote recording requirements address the enclosure, equipment,
and supplies used by voters to vote.
3.2.4.1 Common Standards
All systems shall provide voting booths or enclosures for poll site
use. Such booths or enclosures may be integral to the voting system or
supplied as components of the voting system, and shall:
a. Be integral to, or makes provision for, the installation of, the
voting device;
b. Ensure by its structure stability against movement or
overturning during entry, occupancy, and exit by the voter;
c. Provide privacy for the voter, and be designed in such a way as
to prevent observation of the ballot by any person other than the
voter; and
d. Be capable of meeting the accessibility requirements of Section
2.2.7.1.
3.2.4.2 Paper-Based Recording Standards
The paper-based recording requirements govern:
Ballot cards or sheets, and pages or assemblies of pages
containing ballot field identification data;
Punching devices;
Marking devices;
Frames or fixtures to hold the ballot while it is being
punched;
Compartments or booths where voters record selections; and
Secure containers for the collection of voted ballots.
3.2.4.2.1 Paper Ballot Standards
Paper ballots used by paper-based voting systems shall meet the
following standards:
a. Punches or marks that identify the unique ballot format, in
accordance with Section 2.3.1.1.1.c., shall be

[[Page 18962]]

outside the area in which votes are recorded, so as to minimize the
likelihood that these punches or marks will be mistaken for vote
responses and the likelihood that recorded votes will obliterate these
punches or marks;
b. If printed or punched alignment marks are used to locate the
vote response fields on the ballot, these marks shall be outside the
area in which votes are recorded, so as to minimize the likelihood that
these marks will be mistaken for vote responses and the likelihood that
recorded votes will obliterate these marks; and
c. The TDP shall specify the required paper stock, size, shape,
opacity, color, watermarks, field layout, orientation, size and style
of printing, size and location of punch or mark fields used for vote
response fields and to identify unique ballot formats, placement of
alignment marks, ink for printing, and folding and bleed-through
limitations for preparation of ballots that are compatible with the
system.
3.2.4.2.2 Punching Devices
Punching devices used by voting systems shall:
a. Be suitable for the type of ballot card specified;
b. Facilitate the clear and accurate recording of each vote
intended by the voter;
c. Be designed to avoid excessive damage to vote recorder
components; and
d. Incorporate features to ensure that the chad (debris) is
completely removed, without damage to other parts of the ballot card.
3.2.4.2.3 Marking Devices
The TDP shall specify marking devices (such as pens or pencils)
that, if used to make the prescribed form of mark, produce readable
marked ballots such that the system meets the performance requirements
for accuracy specified previously. These specifications shall identify:
a. Specific characteristics of marking devices that affect
readability of marked ballots;
b. Performance capabilities with regard to each characteristic; and
c. For marking devices manufactured by multiple external sources, a
listing of sources and model numbers that are compatible with the
system.
3.2.4.2.4 Frames or Fixtures for Punchcard Ballots
The frame or fixture for punchcards shall:
a. Hold the ballot card securely in its proper location and
orientation for voting;
b. When contests are not printed directly on the ballot card or
sheet, incorporate an assembly of ballot label pages that identify the
offices and issues corresponding to the proper ballot format for the
polling place where it is used and that are aligned with the voting
fields assigned to them; and
c. Incorporate a template to preclude perforation of the card
except in the specified voting fields; a mask to allow punches only in
fields designated by the format of the ballot; and a backing plate for
the capture and removal of chad. This requirement may be satisfied by
equipment of a different design as long it achieves the same result as
the Standards with regard to:
(1) Positioning the card;
(2) Association of ballot label information with corresponding
punch fields;
(3) Enabling of only those voting fields that correspond to the
format of the ballot; and
(4) Punching the fields and the positive removal of chad.
3.2.4.2.5 Frames or Fixtures for Printed Ballots
A frame or fixture for printed ballot cards is optional. However,
if such a device is provided, it shall:
a. Be of any size and shape consistent with its intended use;
b. Position the card properly;
c. Hold the ballot card securely in its proper location and
orientation for voting; and
d. Comply with the requirements for design and construction
contained in Section 3.4.
3.2.4.2.6 Ballot Boxes and Ballot Transfer Boxes
Ballot boxes and ballot transfer boxes, which serve as secure
containers for the storage and transportation of voted ballots, shall:
a. Be of any size, shape, and weight commensurate with their
intended use;
b. Incorporate locks or seals, the specifications of which are
described in the system documentation;
c. Provide specific points where ballots are inserted, with all
other points on the box constructed in a manner that prevents ballot
insertion; and
d. For precinct count systems, contain separate compartments for
the segregation of unread ballots, ballots containing write-in votes,
or any irregularities that may require special handling or processing.
In lieu of compartments, the conversion processing may mark such
ballots with an identifying spot or stripe to facilitate manual
segregation.
3.2.4.3 DRE Systems Recording Requirements
The DRE systems recording requirements address the detection and
recording of votes, including the logic and data processing functions
required to determine the validity of voter selections, to accept and
record valid selections, and to reject invalid selections. The
requirements also address the physical environment in which ballots are
cast.
3.2.4.3.1 Activity Indicator
DRE systems shall include an audible or visible activity indicator
providing the status of each voting device. This indicator shall:
a. Indicate whether the device has been activated for voting; and
b. Indicate whether the device is in use.
3.2.4.3.2 DRE System Vote Recording
To ensure vote recording accuracy and integrity while protecting
the anonymity of the voter, all DRE systems shall:
a. Contain all mechanical, electromechanical, and electronic
components; software; and controls required to detect and record the
activation of selections made by the voter in the process of voting and
casting a ballot;
b. Incorporate redundant memories to detect and allow correction of
errors caused by the failure of any of the individual memories;
c. Provide at least two processes that record the voter's
selections that:
(1) To the extent possible, are isolated from each other;
(2) Designate one process and associated storage location as the
main vote detection, interpretation, processing and reporting path; and
(3) Use a different process to store ballot images, for which the
method of recording may include any appropriate encoding or data
compression procedure consistent with the regeneration of an
unequivocal record of the ballot as cast by the voter.
d. Provide a capability to retrieve ballot images in a form
readable by humans; and
e. Ensure that all processing and storage protects the anonymity of
the voter.
3.2.4.3.3 Recording Accuracy
DRE systems shall meet the following requirements for recording
accurately each vote and ballot cast:
a. Detect every selection made by the voter;

[[Page 18963]]

b. Correctly add permissible selections to the memory components of
the device;
c. Verify the correctness of the detection of the voter selections
and the addition of the selections to memory;
d. Achieve an error rate not to exceed the requirement indicated in
Section 3.2.1;
e. Preserve the integrity of voting data and ballot images (for DRE
machines) stored in memory for the official vote count and audit trail
purposes against corruption by stray electromagnetic emissions, and
internally generated spurious electrical signals; and
f. Maintain a log of corrected data.
3.2.4.3.4 Recording Reliability
Recording reliability refers to the ability of the DRE system to
record votes accurately at its maximum rated processing volume for a
specified period of time. The DRE system shall record votes reliably in
accordance with the requirements of Section 3.4.3.
3.2.5 Paper-Based Conversion Requirements
The paper-based conversion requirements address the ability of the
system to read the ballot card and to translate its pattern of punches
or marks into electronic signals for later processing. These
capabilities may be built into the voting system in an integrated
fashion, or may be provided by one or more components that are not
unique to the system, such as a general-purpose data processing card
reader or read head suitably interfaced to the system. These
requirements address two major functions: ballot handling and ballot
reading.
3.2.5.1 Ballot Handling
Ballot handling consists of a ballot card's acceptance, movement
through the read station, and transfer into a collection station or
receptacle.
3.2.5.1.1 Capacity (Central Count)
The capacity to convert the punches or marks on individual ballots
into signals is uniquely important to central count systems. The
capacity for a central count system shall be documented by the vendor.
This documentation shall include the capacity for individual components
that impact the overall capacity.
3.2.5.1.2 Exception Handling (Central Count)
This requirement refers to the handling of ballots for a central
count system when they are unreadable or when some condition is
detected requiring that the cards be segregated from normally processed
ballots for human review. In response to an unreadable ballot or a
write-in vote all central count paper-based systems shall:
a. Outstack the ballot, or
b. Stop the ballot reader and display a message prompting the
election official or designee to remove the ballot, or
c. Mark the ballot with an identifying mark to facilitate its later
identification.
Additionally, the system shall provide a capability that can be
activated by an authorized election official to identify ballots
containing overvotes, blank ballots, and ballots containing undervotes
in a designated race. If enabled, these capabilities shall perform one
of the above actions in response to the indicated condition.
3.2.5.1.3 Exception Handling (Precinct Count)
This requirement refers to the handling of ballots for a precinct
count system when they are unreadable or when some condition is
detected requiring that the cards be segregated from normally processed
ballots for human review. All paper based precinct count systems shall:
a. In response to an unreadable or blank ballot, return the ballot
and provide a message prompting the voter to examine the ballot;
b. In response to a ballot with a write-in vote, segregate the
ballot or mark the ballot with an identifying mark to facilitate its
later identification;
c. In response to a ballot with an overvote the system shall:
(1) Provide a capability to identify an overvoted ballot;
(2) Return the ballot;
(3) Provide an indication prompting the voter to examine the
ballot;
(4) Allow the voter to submit the ballot with the overvote; and
(5) Provide a means for an authorized election official to
deactivate this capability entirely and by contest; and
d. In response to a ballot with an undervote the system shall:
(1) Provide a capability to identify an undervoted ballot;
(2) Return the ballot;
(3) Provide an indication prompting the voter to examine the
ballot;
(4) Allow the voter to submit the ballot with the undervote; and
(5) Provide a means for an authorized election official to
deactivate this capability.
3.2.5.1.4 Multiple Feed Prevention
Multiple feed refers to the situation arising when a ballot reader
attempts to read more than one ballot at a time. The requirements
govern the ability of a ballot reader to prevent multiple feed or to
detect and provide an alarm indicating multiple feed.
a. If multiple feed is detected, the card reader shall halt in a
manner that permits the operator to remove the unread cards causing the
error, and reinsert them in the card input hopper.
b. The frequency of multiple feeds with ballots intended for use
with the system shall not exceed 1 in 10,000.
3.2.5.2 Ballot Reading Accuracy
This paper-based system requirement governs the conversion of the
physical ballot into electronic data. Reading accuracy for ballot
conversion refers to the ability to:
Recognize vote punches or marks, or the absence thereof,
for each possible selection on the ballot;
Discriminate between valid punches or marks and extraneous
perforations, smudges, and folds; and
Convert the vote punches or marks, or the absence thereof,
for each possible selection on the ballot into digital signals.
To ensure accuracy, paper-based systems shall:
a. Detect punches or marks that conform to vendor specifications
with an error rate not exceeding the requirement indicated in Section
3.2.1;
b. Ignore, and not record, extraneous perforations, smudges, and
folds; and
c. Reject ballots that meet all vendor specifications at a rate not
to exceed 2 percent.
3.2.6 Processing Requirements
Processing requirements apply to the hardware and software required
to accumulate voting data for all candidates and measures within voting
machines and polling places, and to consolidate the voting data at a
central level or multiple levels. These requirements also address the
generation and maintenance of audit records, the detection and
disabling of improper use or operation of the system, and the
monitoring of overall system status. Separate and distinct requirements
for paper-based and DRE voting systems are presented below.
3.2.6.1 Paper-Based System Processing Requirements
The paper-based processing requirements address all mechanical
devices, electromechanical devices, electronic devices, and software
required to perform the logical and numerical functions of interpreting
the electronic image of the voted ballot, and assigning votes to the
proper memory registers.

[[Page 18964]]

3.2.6.1.1 Processing Accuracy
Processing accuracy refers to the ability of the system to receive
electronic signals produced by punches for punchcard systems and vote
marks and timing information for marksense systems; perform logical and
numerical operations upon these data; and reproduce the contents of
memory when required, without error. Specific requirements are detailed
below:
a. Processing accuracy shall be measured by vote selection error
rate, the ratio of uncorrected vote selection errors to the total
number of ballot positions that could be recorded across all ballots
when the system is operated at its nominal or design rate of
processing;
b. The vote selection error rate shall include data that denotes
ballot style or precinct as well as data denoting a vote in a specific
contest or ballot proposition;
c. The vote selection error rate shall include all errors from any
source; and
d. The vote selection error rate shall not exceed the requirement
indicated in Section 3.2.1.
3.2.6.1.2 Memory Stability
Paper-based system memory devices, used to retain control programs
and data, shall have demonstrated error-free data retention for a
period of 22 months, under the environmental conditions for operation
and non-operation (i.e. storage).
3.2.6.2 DRE System Processing Requirements
The DRE system processing requirements address all mechanical
devices, electromechanical devices, electronic devices, and software
required to process voting data after the polling places are closed.
3.2.6.2.1 Processing Speed
DRE voting systems shall meet the following requirements for
processing speed:
a. Operate at a speed sufficient to respond to any operator and
voter input without perceptible delay (no more than three seconds); and
b. If the consolidation of polling place data is done locally,
perform this consolidation in a time not to exceed five minutes for
each device in the polling place.
3.2.6.2.2 Processing Accuracy
Processing accuracy is defined as the ability of the system to
process voting data stored in DRE voting devices, or in removable
memory modules installed in such devices. Processing includes all
operations to consolidate voting data after the polling places have
been closed. DRE voting systems shall:
a. Produce reports that are completely consistent, with no
discrepancy among reports of voting device data produced at any level;
and
b. Produce consolidated reports containing absentee, provisional,
or other voting data that are similarly error-free. Any discrepancy,
regardless of source, is resolvable to a procedural error, to the
failure of a non-memory device, or to an external cause.
3.2.6.2.3 Memory Stability
DRE system memory devices used to retain control programs and data
shall have demonstrated error-free data retention for a period of 22
months. Error-free retention may be achieved by the use of redundant
memory elements, provided that the capability for conflict resolution
or correction among elements is included.
3.2.7 Reporting Requirements
The reporting requirements govern all mechanical,
electromechanical, and electronic devices required for voting systems
to print audit record entries and results of the tabulation. These
requirements also address data storage media for transportation of data
to other sites.
3.2.7.1 Removable Storage Media
In voting systems that use storage media that can be removed from
the system and transported to another location for readout and report
generation, these media shall use devices with demonstrated error-free
retention for a period of 22 months under the environmental conditions
for operation and non-operation contained in Section 3.2.2. Examples of
removable storage media include: programmable read-only memory (PROM),
random access memory (RAM) with battery backup, magnetic media, or
optical media.
3.2.7.2 Printers
All printers used to produce reports of the vote count shall be
capable of producing:
a. Alphanumeric headers;
b. Election, office and issue labels; and
c. Alphanumeric entries generated as part of the audit record.
3.2.8 Vote Data Management Requirements
The vote data management requirements for all systems address
capabilities that manage, process, and report voting data after the
data has been consolidated at the polling place or other intermediate
levels. These capabilities allow the system to:
a. Consolidate voting data from polling place data memory or
transfer devices;
b. Report polling place summaries; and
c. Process absentee ballots, data entered manually, and
administrative ballot definition data.
The requirements address all hardware and software required to
generate output reports in the various formats required by the using
jurisdiction.
3.2.8.1 Data File Management
All voting systems shall provide the capability to:
a. Integrate voting data files with ballot definition files;
b. Verify file compatibility; and
c. Edit and update files as required.
3.2.8.2 Data Report Generation
All voting systems shall include report generators for producing
output reports at the device, polling place, and summary level, with
provisions for administrative and judicial subdivisions as required by
the using jurisdiction.
3.3 Physical Characteristics
This section covers physical characteristics of all voting systems
and components that affect their general utility and suitability for
election operations.
3.3.1 Size
There is no numerical limitation on the size of any voting system
equipment, but the size of each device should be compatible with its
intended use and the location at which the equipment is to be used.
3.3.2 Weight
There is no numerical limitation on the weight of any voting system
equipment, but the weight of each device should be compatible with its
intended use and the location at which the equipment is to be used.
3.3.3 Transport and Storage of Precinct Systems
All precinct systems shall:
a. Provide a means to safely and easily handle, transport, and
install polling place equipment, such as wheels or a handle or handles;
and
b. Be capable of using, or be provided with, a protective enclosure
rendering the equipment capable of withstanding:
(1) Impact, shock and vibration loads accompanying surface and air
transportation; and
(2) Stacking loads accompanying storage.

[[Page 18965]]

3.4 Design, Construction, and Maintenance Characteristics
This section covers voting system materials, construction
workmanship, and specific design characteristics important to the
successful operation and efficient maintenance of the system.
3.4.1 Materials, Processes, and Parts
The approach to system design is unrestricted, and may incorporate
any form or variant of technology capable of meeting the voting systems
requirements and standards.
Precinct count systems shall be designed in accordance with best
commercial practice for microcomputers, process controllers, and their
peripheral components. Central count voting systems and equipment used
in a central tabulating environment shall be designed in accordance
with best commercial and industrial practice.
All voting systems shall:
a. Be designed and constructed so that the frequency of equipment
malfunctions and maintenance requirements are reduced to the lowest
level consistent with cost constraints;
b. Include, as part of the accompanying TDP, an approved parts
list; and
c. Exclude parts or components not included in the approved parts
list.
3.4.2 Durability
All voting systems shall be designed to withstand normal use
without deterioration and without excessive maintenance cost for a
period of ten years.
3.4.3 Reliability
The reliability of voting system devices shall be measured as mean
time between Failure (MTBF) for the system submitted for testing. MBTF
is defined as the value of the ratio of operating time to the number of
failures which have occurred in the specified time interval. A typical
system operations scenario consist of approximately 45 hours of
equipment operation, consisting of 30 hours of equipment set-up and
readiness testing and 15 hours of elections operations. For the purpose
of demonstrating compliance with this requirement, a failure is defined
as any event which results in either the:
a. Loss of one or more functions; or
b. Degradation of performance such that the device is unable to
perform its intended function for longer than 10 seconds.
The MTBF demonstrated during qualification testing shall be at
least 163 hours.
3.4.4 Maintainability
Maintainability represents the ease with which maintenance actions
can be performed based on the design characteristics of equipment and
software and the processes the vendor and election officials have in
place for preventing failures and for reacting to failures.
Maintainability includes the ability of equipment and software to self-
diagnose problems and make non-technical election workers aware of a
problem. Maintainability addresses all scheduled and unscheduled
events, which are performed to:
Determine the operational status of the system or a
component;
Adjust, align, tune, or service components;
Repair or replace a component having a specified operating
life or replacement interval;
Repair or replace a component that exhibits an undesirable
predetermined physical condition or performance degradation;
Repair or replace a component that has failed; and
Verify the restoration of a component, or the system, to
operational status.
Maintainability shall be determined based on the presence of
specific physical attributes that aid system maintenance activities,
and the ease with which system maintenance tasks can be performed by
the ITA. Although a more quantitative basis for assessing
maintainability, such as the mean to repair the system is desirable,
the qualification of a system is conducted before it is approved for
sale and thus before a broader base of maintenance experience can be
obtained.
3.4.4.1 Physical Attributes
The following physical attributes will be examined to assess
reliability:
a. Presence of labels and the identification of test points;
b. Provision of built-in test and diagnostic circuitry or physical
indicators of condition;
c. Presence of labels and alarms related to failures; and
d. Presence of features that allow non-technicians to perform
routine maintenance tasks (such as update of the system database).
3.4.4.2 Additional Attributes
The following additional attributes will be considered to assess
system maintainability.
a. Ease of detecting that equipment has failed by a non-technician;
b. Ease of diagnosing problems by a trained technician;
c. Low false alarm rates (i.e., indications of problems that do not
exist);
d. Ease of access to components for replacement;
e. Ease with which adjustment and alignment can be performed;
f. Ease with which database updates can be performed by a non-
technician; and
g. Adjust, align, tune, or service components.
3.4.5 Availability
The availability of a voting system is defined as the probability
that the equipment (and supporting software) needed to perform
designated voting functions will respond to operational commands and
accomplish the function. The voting system shall meet the availability
standard for each of the following voting functions:
a. For all paper-based systems:
(1) Recording voter selections (such as by ballot marking or
punch); and
(2) Scanning the punches or marks on paper ballots and converting
them into digital data;
b. For all DRE systems, recording and storing the voter's ballot
selections.
c. For precinct-count systems (paper-based and DRE), consolidation
of vote selection data from multiple precinct-based systems to generate
jurisdiction-wide vote counts, including storage and reporting of the
consolidated vote data; and
d. For central-count systems (paper-based and DRE), consolidation
of vote selection data from multiple counting devices to generate
jurisdiction-wide vote counts, including storage and reporting of the
consolidated vote data.
System availability is measured as the ratio of the time during
which the system is operational a (up time) to the total time period of
operation (up time plus down time). Inherent availability (Ai) is a the
fraction of time a system is functional, based upon Mean Time Between
Failure (MTBF) and Mean Time to Repair (MTTR), that is:
Ai = (MTBF)/(MTBF + MTTR)
Mean Time to Repair (MTTR) is the average time required to perform
a corrective maintenance task during periods of system operation.
Corrective maintenance task time is active repair time, plus the time
attributable to other factors that could lead to logistic or
administrative delays, such as travel notification of qualified
maintenance personnel and travel time for such personnel to arrive at
the appropriate site.
Corrective maintenance may consist of substitution of the complete
device or one of its components, as in the case of precinct count and
some central count systems, or it may consist of on-site repair.

[[Page 18966]]

The voting system shall achieve at least ninety nine percent
availability during normal operation for the functions indicated above.
This standard encompasses for each function the combination of all
devices and components that support the function, including their MTTR
and MTBF attribute.
Vendors shall specify the typical system configuration that is to
be used to assess availability, and any assumptions made with regard to
any parameters that impact the MTTR. These factors shall include at a
minimum:
a. Recommended number and locations of spare devices or components
to be kept on hand for repair purposes during periods of system
operation;
b. Recommended number and locations of qualified maintenance
personnel who need to be available to support repair calls during
system operation; and
c. Organizational affiliation (i.e., jurisdiction, vendor) of
qualified maintenance personnel.
3.4.6 Product Marking
All voting systems shall:
a. Identify all devices by means of a permanently affixed nameplate
or label containing the name of the manufacturer or vendor, the name of
the device, its part or model number, its revision letter, its serial
number, and if applicable, its power requirements;
b. Display on each device a separate data plate containing a
schedule for and list of operations required to service or to perform
preventive maintenance; and
c. Display advisory caution and warning instructions to ensure safe
operation of the equipment and to avoid exposure to hazardous
electrical voltages and moving parts at all locations where operation
or exposure may occur.
3.4.7 Workmanship
To help ensure proper workmanship, all manufacturers of voting
systems shall:
a. Adopt and adhere to practices and procedures to ensure that
their products are free from damage or defect that could make them
unsatisfactory for their intended purpose; and
b. Ensure that components provided by external suppliers are free
from damage or defect that could make them unsatisfactory for their
intended purpose.
3.4.8 Safety
All voting systems shall meet the following requirements for
safety:
a. All voting systems and their components shall be designed so as
to eliminate hazards to personnel, or to the equipment itself;
b. Defects in design and construction that can result in personal
injury or equipment damage must be detected and corrected before voting
systems and components are placed into service; and
c. Equipment design for personnel safety shall be equal to or
better than the appropriate requirements of the Occupational Safety and
Health Act (OSHA), as identified in Title 29, part 1910, of the Code of
Federal Regulations.

Volume I, Section 4

Table of Contents

4 Software Standards
4.1 Scope
4.1.1 Software Sources
4.1.2 Location and Control of Software and Hardware on Which it
Operates
4.1.3 Exclusions
4.2 Software Design and Coding Standards
4.2.1 Selection of Programming Languages
4.2.2 Software Integrity
4.2.3 Software Modularity and Programming
4.2.4 Control Constructs
4.2.5 Naming Conventions
4.2.6 Coding Conventions
4.2.7 Comment Conventions
4.3 Data and Document Retention
4.4 Audit Record Data
4.4.1 Pre-election Audit Records
4.4.2 System Readiness Audit Records
4.4.3 In-Process Audit Records
4.4.4 Vote Tally Data
4.5 Vote Secrecy (DRE Systems)

4 Software Standards

4.1 Scope

This section describes essential design and performance
characteristics of the software used in voting systems, addressing both
system-level software, such as operating systems, and voting system
application software, including firmware. The requirements of this
section are intended to ensure that voting system software is reliable,
robust, testable, and maintainable. The standards in this section also
support system accuracy, logical correctness, privacy, security and
integrity.
The general requirements of this section apply to software used to
support the entire range of voting system activities described in
Section 2. More specific requirements are defined for ballot counting,
vote processing, creating an audit trail, and generating output reports
and files. Although this section emphasizes software, the standards
described also influence hardware design considerations.
This section recognizes that there is no best way to design
software. Many programming languages are available for which modern
programming practices are applicable, such as the use of rigorous
program and data structures, data typing, and naming conventions. Other
programming languages exist for which such practices are not easily
applied.
The Standards are intended to guide the design of software written
in any of the programming languages commonly used for mainframe, mini-
computer, and microprocessor systems. They are not intended to preclude
the use of other languages or environments, such as those that exhibit
``declarative'' structure, ``object-oriented'' languages,
``functional'' programming languages, or any other combination of
language and implementation that provides appropriate levels of
performance, testability, reliability, and security. The vendor makes
specific software selections. However, the use of widely recognized and
proven software design methods will facilitate the analysis and testing
of voting system software in the qualification process.
4.1.1 Software Sources
The requirements of this section apply generally to all software
used in voting systems, including:
Software provided by the voting system vendor and its
component suppliers;
Software furnished by an external provider (for example,
providers of COTS operating systems and web browsers) where the
software may be used in any way during voting system operation; and
Software developed by the voting jurisdiction.
Compliance with the requirements of the software standards is
assessed by several formal tests, including code examination.
Unmodified software is not subject to code examination; however, source
code generated by a package and embedded in software modules for
compilation or interpretation shall be provided in human readable form
to the ITA. The ITA may inspect source code units to determine testing
requirements or to verify that the code is unmodified and that the
default configuration options have not been changed.
Configuration of software, both operating systems and applications,
is critical to proper system functioning. Correct test design and
sufficient test execution must account for the intended and proper
configuration of all system components. Therefore, the vendors shall
submit to the ITA, in the TDP, a

[[Page 18967]]

record of all user selections made during software installation. The
vendor shall also submit a record of all configuration changes made to
the software following its installation. The ITA shall confirm the
propriety and correctness of these user selections and configuration
changes.
4.1.2 Location and Control of Software and Hardware on Which it
Operates
The requirements of this section apply to all software used in any
manner to support any voting-related activities, regardless of the
ownership of the software or the ownership and location of the hardware
on which the software is installed or operates. These requirements
apply to:
Software that operates on voting devices and vote counting
devices installed at polling places under the control of the voting
jurisdiction;
Software that operates on ballot printers, vote counting
devices, and other hardware typically installed at central or precinct
locations (including contractor facilities); and
Election management software.
However, some requirements apply only in specific situations
indicated in this section. In addition to the requirements of this
section, all software used in any manner to support any voting-related
activities shall meet the requirements for security described in
Section 6 of the Standards.
4.1.3 Exclusions
Some voting systems use equipment, such as personal computers, that
may be used for other purposes and have resident on the equipment
general purpose software such as operating systems, programming
language compilers, database management systems, and Web browsers. Such
software is governed by the Standards unless:
The software provides no support of voting system
capabilities;
The software is removable, disconnectable, or switchable
such that it cannot function while voting system functions are enabled;
and
Procedures are provided that confirm that the software has
been removed, disconnected, or switched.

4.2 Software Design and Coding Standards

The software used by voting systems is selected by the vendor and
not prescribed by the Standards. This section provides standards for
voting system software with regard to:
Selection of programming languages;
Software integrity;
Software modularity and programming;
Control constructs;
Naming conventions;
Coding conventions; and
Comment conventions.
4.2.1 Selection of Programming Languages
Software associated with the logical and numerical operations of
vote data shall use a high-level programming language, such as: Pascal,
Visual Basic, Java, C and C++. The requirement for the use of high-
level language for logical operations does not preclude the use of
assembly language for hardware-related segments, such as device
controllers and handler programs. Also, operating system software may
be designed in assembly language.
4.2.2 Software Integrity
Self-modifying, dynamically loaded, or interpreted code is
prohibited, except under the security provisions outlined in section
6.4.e. This prohibition is to ensure that the software tested and
approved during the qualification process remains unchanged and retains
its integrity. External modification of code during execution shall be
prohibited. Where the development environment (programming language and
development tools) includes the following features, the software shall
provide controls to prevent accidental or deliberate attempts to
replace executable code:
Unbounded arrays or strings (includes buffers used to move
data);
Pointer variables; and
Dynamic memory allocation and management.
4.2.3 Software Modularity and Programming
Voting system application software, including COTS software, shall
be designed in a modular fashion. However, COTS software is not
required to be inspected for compliance with this requirement. For the
purpose of this requirement \1\, ``modules'' may be compiled or
interpreted independently. Modules may also be nested. The modularity
rules described here apply to the component sub modules of a library.
The principle concept is that the module contains all the elements to
compile or interpret successfully and has limited access to data in
other modules. The design concept is simple replacement with another
module whose interfaces match the original module. A module is designed
in accordance with the following rules:
---------------------------------------------------------------------------

\1\ Some software languages and development environments use a
different definition of module but this principle still applies.
---------------------------------------------------------------------------

a. Each module shall have a specific function that can be tested
and verified independently of the remainder of the code. In practice,
some additional modules (such as library modules) may be needed to
compile the module under test, but the modular construction allows the
supporting modules to be replaced by special test versions that support
test objectives;
b. Each module shall be uniquely and mnemonically named, using
names that differ by more than a single character. In addition to the
unique name, the modules shall include a set of header comments
identifying the module's purpose, design, conditions, and version
history, followed by the operational code. Headers are optional for
modules of fewer than ten executable lines where the subject module is
embedded in a larger module that has a header containing the header
information. Library modules shall also have a header comment
describing the purpose of the library and version information;
c. All required resources, such as data accessed by the module,
should either be contained within the module or explicitly identified
as input or output to the module. Within the constraints of the
programming language, such resources shall be placed at the lowest
level where shared access is needed. If that shared access level is
across multiple modules, the definitions should be defined in a single
file (called header files in some languages, such as C) where any
changes can be applied once and the change automatically applies to all
modules upon compilation or activation;
d. A module is small enough to be easy to follow and understand.
Program logic visible on a single page is easy to follow and correct.
Volume II, Section 5 provides testing guidelines for the ITA to
identify large modules subject to review under this requirement;
e. Each module shall have a single entry point, and a single exit
point, for normal process flow. For library modules or languages such
as the object-oriented languages, the entry point is to the individual
contained module or method invoked. The single exit point is the point
where control is returned. At that point, the data that is expected as
output must be appropriately set. The exception for the exit point is
where a problem is so severe that execution cannot be resumed. In this
case, the design must explicitly protect all recorded votes and audit
log information and must implement formal exception handlers provided
by the language; and

[[Page 18968]]

f. Process flow within the modules shall be restricted to
combinations of the control structures defined in Volume II, Section 5.
These structures support the modular concept, especially the single
entry/exit rule above. They apply to any language feature where program
control passes from one activity to the next, such as control scripts,
object methods, or sets of executable statements, even though the
language itself is not procedural.
4.2.4 Control Constructs
Voting system software shall use the control constructs identified
in Volume II, Section 5:
a. Acceptable constructs are Sequence, If-Then-Else, Do-While, Do-
Until, Case, and the General loop (including the special case for
loop);
b. If the programming language used does not provide these control
constructs, the vendor shall provide them (that is, comparable control
structure logic). The constructs shall be used consistently throughout
the code. No other constructs shall be used to control program logic
and execution;
c. While some programming languages do not create programs as
linear processes, stepping from an initial condition, through changes,
to a conclusion, the program components nonetheless contain procedures
(such as ``methods'' in object-oriented languages). Even in these
programming languages, the procedures must execute through these
control constructs (or their equivalents, as defined and provided by
the vendor); and
d. Operator intervention or logic that evaluates received or stored
data shall not re-direct program control within a program routine.
Program control may be re-directed within a routine by calling
subroutines, procedures, and functions, and by interrupt service
routines and exception handlers (due to abnormal error conditions). Do-
While (False) constructs and intentional exceptions (used as GoTos) are
prohibited.
4.2.5 Naming Conventions
Voting system software shall use the following naming conventions:
a. Object, function, procedure, and variable names shall be chosen
so as to enhance the readability and intelligibility of the program.
Insofar as possible, names shall be selected so that their parts of
speech represent their use, such as nouns to represent objects, verbs
to represent functions, etc.;
b. Names used in code and in documentation shall be consistent;
c. Names shall be unique within an application. Names shall differ
by more than a single character. All single-character names are
forbidden except those for variables used as loop indexes. In large
systems where subsystems tend to be developed independently, duplicate
names may be used where the scope of the name is unique within the
application. Names should always be unique where modules are shared;
and
d. Language keywords shall not be used as names of objects,
functions, procedures, variables, or in any manner not consistent with
the design of the language.
4.2.6 Coding Conventions
Voting system software shall adhere to basic coding conventions.
The coding conventions used shall meet one of the following conditions:
a. The vendors shall identify the published, reviewed, and
industry-accepted coding conventions used and the ITAs shall test for
compliance; or
b. The ITAs shall evaluate the code using the coding convention
requirements specified in Volume II, Section 5.
These standards reference conventions that protect the integrity
and security of the code, which may be language-specific, and language-
independent conventions that significantly contribute to readability
and maintainability. Specific style conventions that support economical
testing are not binding unless adopted by the vendor.
4.2.7 Comment Conventions
Voting system software shall use the following comment conventions:
a. All modules shall contain headers. For small modules of 10 lines
or less, the header may be limited to identification of unit and
revision information. Other header information should be included in
the small unit headers if not clear from the actual lines of code.
Header comments shall provide the following information:
(1) The purpose of the unit and how it works;
(2) Other units called and the calling sequence;
(3) A description of input parameters and outputs;
(4) File references by name and method of access (read, write,
modify, append, etc.);
(5) Global variables used; and
(6) Date of creation and a revision record;
b. Descriptive comments shall be provided to identify objects and
data types. All variables shall have comments at the point of
declaration clearly explaining their use. Where multiple variables that
share the same meaning are required, the variables may share the same
comment;
c. In-line comments shall be provided to facilitate interpretation
of functional operations, tests, and branching;
d. Assembly code shall contain descriptive and informative
comments, such that its executable lines can be clearly understood; and
e. All comments shall be formatted in a uniform manner that makes
it easy to distinguish them from executable code.

4.3 Data and Document Retention

All systems shall:
a. Maintain the integrity of voting and audit data during an
election, and for at least 22 months thereafter, a time sufficient in
which to resolve most contested elections and support other activities
related to the reconstruction and investigation of a contested
election; and
b. Protect against the failure of any data input or storage device
at a location controlled by the jurisdiction or its contractors, and
against any attempt at improper data entry or retrieval.

4.4 Audit Record Data

Audit trails are essential to ensure the integrity of a voting
system. Operational requirements for audit trails are described in
Section 2.2.5.2 of the Standards. Audit record data are generated by
these procedures. The audit record data in the following subsections
are essential to the complete recording of election operations and
reporting of the vote tally. This list of audit records may not reflect
the design constructs of some systems. Therefore, vendors shall
supplement it with information relevant to the operation of their
specific systems.
4.4.1 Pre-Election Audit Records
During election definition and ballot preparation, the system shall
audit the preparation of the baseline ballot formats and modifications
to them, a description of these modifications, and corresponding dates.
The log shall include:
a. The allowable number of selections for an office or issue;
b. The combinations of voting patterns permitted or required by the
jurisdiction;
c. The inclusion or exclusion of offices or issues as the result of
multiple districting within the polling place;
d. Any other characteristics that may be peculiar to the
jurisdiction, the election, or the polling place's location;
e. Manual data maintained by election personnel;
f. Samples of all final ballot formats; and

[[Page 18969]]

g. Ballot preparation edit listings.
4.4.2 System Readiness Audit Records
The following minimum requirements apply to system readiness audit
records:
a. Prior to the start of ballot counting, a system process shall
verify hardware and software status and generate a readiness audit
record. This record shall include the identification of the software
release, the identification of the election to be processed, and the
results of software and hardware diagnostic tests;
b. In the case of systems used at the polling place, the record
shall include the polling place's identification;
c. The ballot interpretation logic shall test and record the
correct installation of ballot formats on voting devices;
d. The software shall check and record the status of all data paths
and memory locations to be used in vote recording to protect against
contamination of voting data;
e. Upon the conclusion of the tests, the software shall provide
evidence in the audit record that the test data have been expunged;
f. If required and provided, the ballot reader and arithmetic-logic
unit shall be evaluated for accuracy, and the system shall record the
results. It shall allow the processing, or simulated processing, of
sufficient test ballots to provide a statistical estimate of processing
accuracy; and
g. For systems that use a public network, provide a report of test
ballots that includes:
(1) Number of ballots sent;
(2) When each ballot was sent;
(3) Machine from which each ballot was sent; and
(4) Specific votes or selections contained in the ballot.
4.4.3 In-Process Audit Records
In-process audit records document system operations during
diagnostic routines and the casting and tallying of ballots. At a
minimum, the in-process audit records shall contain:
a. Machine generated error and exception messages to demonstrate
successful recovery. Examples include, but are not necessarily limited
to:
(1) The source and disposition of system interrupts resulting in
entry into exception handling routines;
(2) All messages generated by exception handlers;
(3) The identification code and number of occurrences for each
hardware and software error or failure;
(4) Notification of system login or access errors, file access
errors, and physical violations of security as they occur, and a
summary record of these events after processing;
(5) Other exception events such as power failures, failure of
critical hardware components, data transmission errors, or other type
of operating anomaly;
b. Critical system status messages other than informational
messages displayed by the system during the course of normal
operations. These items include, but are not limited to:
(1) Diagnostic and status messages upon startup;
(2) The ``zero totals'' check conducted before opening the polling
place or counting a precinct centrally;
(3) For paper-based systems, the initiation or termination of card
reader and communications equipment operation; and
(4) For DRE machines at controlled voting locations, the event (and
time, if available) of activating and casting each ballot (i.e., each
voter's transaction as an event). This data can be compared with the
public counter for reconciliation purposes;
c. Non-critical status messages that are generated by the machine's
data quality monitor or by software and hardware condition monitors;
and
d. System generated log of all normal process activity and system
events that require operator intervention, so that each operator access
can be monitored and access sequence can be constructed.
4.4.4 Vote Tally Data
In addition to the audit requirements described above, other
election-related data is essential for reporting results to interested
parties, the press, and the voting public, and is vital to verifying an
accurate count.
Voting systems shall meet these reporting requirements by providing
software capable of obtaining data concerning various aspects of vote
counting and producing reports of them on a printer. At a minimum, vote
tally data shall include:
a. Number of ballots cast, using each ballot configuration, by
tabulator, by precinct, and by political subdivision;
b. Candidate and measure vote totals for each contest, by
tabulator;
c. The number of ballots read within each precinct and for
additional jurisdictional levels, by configuration, including separate
totals for each party in primary elections;
d. Separate accumulation of overvotes and undervotes for each
contest, by tabulator, precinct and for additional jurisdictional
levels (no overvotes would be indicated for DRE voting devices); and
e. For paper-based systems only, the total number of ballots both
processed and unprocessable; and if there are multiple card ballots,
the total number of cards read.
For systems that produce an electronic file containing vote tally
data, the contents of the file shall include the same minimum data
cited above for printed vote tally reports.

4.5 Vote Secrecy (DRE Systems)

All DRE systems shall ensure vote secrecy by:
a. Immediately after the voter chooses to cast his or her ballot,
record the voter's selections in the memory to be used for vote
counting and audit data (including ballot images), and erase the
selections from the display, memory, and all other storage, including
all forms of temporary storage; and
b. Immediately after the voter chooses to cancel his or her ballot,
erase the selections from the display and all other storage, including
buffers and other temporary storage.

Volume I, Section 5

Table of Contents

5 Telecommunications

5.1 Scope
5.1.1 Types of Components
5.1.2 Telecommunications Operations and Providers
5.1.3 Data Transmissions
5.2 Design, Construction, and Maintenance Requirements
5.2.1 Accuracy
5.2.2 Durability
5.2.3 Reliability
5.2.4 Maintainability
5.2.5 Availability
5.2.6 Integrity
5.2.7 Confirmation

5 Telecommunications

5.1 Scope

This section contains the performance, design, and maintenance
characteristics of the telecommunications components of voting systems
and the acceptable levels of performance against these characteristics.
For the purpose of the Standards, telecommunications is defined as the
capability to transmit and receive data electronically using hardware
and software components over distances both within and external to a
polling place.
The requirements in this section represent acceptable levels of
combined telecommunications hardware and software function and
performance for the transmission of data that is used to operate the
system and report election results. Where applicable, this section
specifies minimum values for critical performance and functional
attributes involving telecommunications hardware and software
components.

[[Page 18970]]

This section does not apply to other means of moving data, such as
the physical transport of data recorded on paper-based media, or the
transport of physical devices, such as memory cards, that store data in
electronic form.
Voting systems may include network hardware and software to
transfer data among systems. Major network components are local area
networks (LANs), wide area networks (WANs), workstations (desktop
computers), servers, data, and applications. Workstations include
voting stations, precinct tabulation systems, and voting supervisory
terminals. Servers include systems that provide registration forms and
ballots and accumulate and process voter registrations and cast
ballots.
Desirable network characteristics include simplicity, flexibility
(especially in routing, to maintain good response times) and
maintainability (including availability, provided primarily through
redundancy of resources and connections, particularly of connections to
public infrastructure).
A wide area network (WAN) public telecommunications component
consists of the hardware and software to transport information, over
shared, public (i.e., commercial or governmental) circuitry, or among
private systems. For voting systems, the telecommunications boundaries
are defined as the transport circuitry, on one side of which exists the
public telecommunications infrastructure, outside the control of voting
system supervisors. On the other side of the transport circuitry are
the local area network (LAN) resources, workstations, servers, data and
applications controlled by voting system supervisors.
Local area network (LAN) components consist of the hardware and
software infrastructure used to transport information between users in
a local environment, typically a building or group of buildings.
Typically a LAN connects workstations, perhaps with a local server.
An application may be a single program or a group of programs that
work together to provide a function to an end user, who may be a voter
or an election administrator. Voter programs may include voter
registration, balloting, and status checking. Administrator programs
may include ballot preparation, registration for preparation,
registration approval, ballot vetting, ballot processing, and election
processing.
This Section is intended to compliment the network security
requirements found in Volume I Section 6, which include requirements
for voter and administrator access, availability of network service,
data confidentiality, and data integrity. Most importantly, security
services will restrict access to local election system components from
public resources, and these services will also restrict access to
voting system data while it is in transit across public resources.
(This is corollary to voting supervisors controlling local election
systems and not assuming control over public resources.)
5.1.1 Types of Components
This section addresses telecommunications hardware and software
across a broad range of technologies including, but not limited to:

--Dial-up communications technologies:
Standard landline;
Wireless;
Microwave;
Very Small Aperture Terminal (VSAT);
Integrated Services Digital Network (ISDN); and
Digital Subscriber Line (DSL);

--High-speed telecommunications lines (public and private):

FT-1, T-1, T-3;
Frame Relay; and
Private line;

--Cabling technologies:

Universal Twisted Pair (UTP) cable (CAT 5 or higher);
Ethernet hub/switch; and
Wireless connections (Radio Frequency (RF) and Infrared);

--Communications routers;
--Modems, whether internal and external to personal computers, computer
servers, and other voting system components (whether installed at the
polling place or central count location);
--Modem drivers, dial-up networking software;
--Channel service units (CSU)/Data service units (DSU) (whether
installed at the polling place or central count location); and
--Dial-up networking applications software.
5.1.2 Telecommunications Operations and Providers
This section applies to voting-related transmissions over public
networks, such as those provided by regional telephone companies and
long distance carriers. This section also applies to private networks
regardless of whether the network is owned and operated by the election
jurisdiction.
For systems that transmit official data over public networks, this
Section applies to telecommunications components installed and operated
at settings supervised by election officials, such as polling places or
central offices. These standards apply to:
Components acquired by the jurisdiction for the purpose of
voting, including components installed at the poll site or a central
office (including central site facilities operated by vendors or
contractors); and
Components acquired by others (such as school systems,
libraries, military installations and other public organizations) that
are used at settings supervised by election officials, including
minimum configuration components required by the vendor but that the
vendor permits to be acquired from third party sources not under the
vendor's control (e.g., router or modem card manufacturer or supplier)
5.1.3 Data Transmissions
These requirements apply to the use of telecommunications to
transmit data for the preparation of the system for an election, the
execution of an election, and the preservation of the system data and
audit trails during and following an election. While this section does
not assume a specific model of voting system operations and does not
assume a specific model for the use of telecommunications to support
such operations, it does address the following types of data, where
applicable:
Voter Authentication: Coded information that confirms the
identity of a voter for security purposes for a system that transmits
votes individually over a public network;
Ballot Definition: Information that describes to a voting
machine the content and appearance of the ballots to be used in an
election;
Vote Transmission: For systems that transmit votes
individually over a public network, the transmission of a single vote
within a network at a polling place and to the county (or contractor)
for consolidation with other county vote data;
Vote Count: Information representing the tabulation of
votes at any level within the control of the jurisdiction, such as the
polling place, precinct, or central count; and
List of Voters: A listing of the individual voters who
have cast ballots in a specific election.
Additional data transmissions used to operate a voting system in
the conduct of an election, but not explicitly listed above, are also
subject to the standards of this section.
For systems that transmit data using public networks, this section
applies to telecommunications hardware and software for transmissions
within and

[[Page 18971]]

among all combinations of senders and receivers indicated below:
Polling places;
Precinct count facilities; and
Central count facilities (whether operated by the
jurisdiction or a contractor).

5.2 Design, Construction, and Maintenance Requirements

Design, construction, and maintenance requirements for
telecommunications represent the operational capability of both system
hardware and software. These capabilities shall be considered basic to
all data transmissions.
5.2.1 Accuracy
The telecommunications components of all voting systems shall meet
the accuracy requirements of Section 3.2.1.
5.2.2 Durability
The telecommunications components of all voting systems shall meet
the durability requirements of Section 3.4.2.
5.2.3 Reliability
The telecommunications components of all voting systems shall meet
the reliability requirements of Section 3.4.3.
5.2.4 Maintainability
The telecommunications components of all voting systems shall meet
the maintainability requirements of Section 3.4.4.
5.2.5 Availability
The telecommunications components of all voting systems shall meet
the availability requirements of Section 3.4.5.
5.2.6 Integrity
For WANs using public telecommunications, boundary definition and
implementation shall meet the following requirements.
a. Outside service providers and subscribers of such providers
shall not be given direct access or control of any resource inside the
boundary;
b. Voting system administrators shall not require any type of
control of resources outside this boundary. Typically, an end point of
a telecommunications circuit will be a subscriber termination on a
Digital Service Unit/Customer Service Unit (DSU/CSU) (though the
precise technology may vary, being such things as cable modems or
routers). Regardless of the technology used, the boundary point must
ensure that everything on one side is locally configured and controlled
while everything on the other side is controlled by an outside service
provider; and
c. The system shall be designed and configured such that it is not
vulnerable to a single point of failure in the connection to the public
network causing total loss of voting capabilities at any polling place.
5.2.7 Confirmation
Confirmation occurs when the system notifies the user of the
successful or unsuccessful completion of the data transmission, where
successful completion is defined as accurate receipt of the transmitted
data. To provide confirmation, the telecommunications components of a
voting system shall:
d. Notify the user of the successful or unsuccessful completion of
the data transmission; and
e. In the event of unsuccessful transmission, notify the user of
the action to be taken.

Volume I, Section 6

Table of Contents

6 Security

6.0 Security
6.0.1 Security Overview (Informative)
6.0.1.1 Independent Dual Verification Systems (Informative)
6.0.1.2 Core characteristics for Independent Verification
Systems (Informative)
6.0.2 Requirements for Voter Verified Paper Audit Trails
(Normative)
6.0.2.1 Display and Print a Paper Record
6.0.2.2 VVPAT Voting Station Usability
6.0.2.3 VVPAT Voting Station Accessibility
6.0.2.4 Approve or Spoil the Paper Record
6.0.2.5 Preserve Voter Privacy and Anonymity
6.0.2.6 Electronic and Paper Record Structure
6.0.2.7 Equipment Security, Reliability, and Maintainability
6.0.3 Wireless Requirements (Normative)
6.0.3.1 Relationship to Volume 1, Section 5:
``Telecommunications''
6.0.3.2 Controlling Usage
6.0.3.3 Identifying Usage
6.0.3.4 Protecting the Transmitted Data
6.0.3.5 Protecting the Wireless Path
6.0.3.6 Protecting the Voting System From a Wireless-based
Attack.
6.0.4 Distribution of Voting System Software and Setup
Validation (Normative)
6.0.4.1 Software Distribution Methodology Requirements
6.0.4.2 Generation and Distribution Requirements for Reference
Information
6.0.4.3 Setup Validation Methodology Requirements
6.1 Scope
6.1.1 System Components and Sources
6.1.2 Location and Control of Software and Hardware on Which it
Operates
6.1.3 Elements of Security Outside Vendor Control
6.1.4 Organization of this Section
6.2 Access Control
6.2.1 Access Control Policy
6.2.1.1 General Access Control Policy
6.2.1.2 Individual Access Privileges
6.2.2 Access Control Measures
6.3 Physical Security Measures
6.3.1 Polling Place Security
6.3.2 Central Count Location Security
6.4 Software Security
6.4.1 Software and Firmware Installation
6.4.2 Protection Against Malicious Software
6.5 Telecommunications and Data Transmission
6.5.1 Access Control
6.5.2 Data Integrity
6.5.3 Data Interception Prevention
6.5.4 Protection Against External Threats
6.5.4.1 Identification of COTS Products
6.5.4.2 Use of Protective Software
6.5.4.3 Monitoring and Responding to External Threats
6.5.5 Shared Operating Environment
6.5.6 Access to Incomplete Election Returns and Interactive
Queries
6.6 Security for Transmission of Official Data Over Public
Communications Networks
6.6.1 General Security Requirements for Systems Transmitting
Data Over Public Networks
6.6.2 Voting Process Security for Casting Individual Ballots
Over a Public Telecommunications Network
6.6.2.1 Documentation of Mandatory Security Activities
6.6.2.2 Capabilities to Operate During Interruption of
Telecommunications Capabilities

6.0 Security

Section 6.0 addresses four new, specific aspects of voting systems
security:
1. Independent Dual Verification Voting Systems: Definition and
characteristics of voting systems that produce multiple records of
votes. A future version of the VVSG will require that voting systems
produce multiple records of ballots or receipts for auditing purposes
(Section 6.0.1, Informative).
2. Security Requirements for Voter Verified Paper Audit Trails:
Requirements for voter verified paper audit trails, if a State chooses
to require them (Section 6.0.2, Normative).
3. Use of Wireless Networking in Voting Systems: Requirements for
wireless networks and the data sent across wireless networks (Section
6.0.3, Normative).
4. Security Requirements for Software Distribution and Setup
Validation of Voting System: Requirements for (a) the secure
distribution of voting systems software and (b) for verifying that
voting systems are operating with the correct software configuration
(Section 6.0.4, Normative).

[[Page 18972]]

1. Security Overview (Informative)

This section is a discussion of independent verification systems
followed by characteristics of independent verification systems which
will be used as the basis for future requirements. The characteristics
are preliminary and will be evolving with further research.
1. Independent Dual Verification Systems
A primary objective for using electronic voting systems is the
production of voting records that are highly precise, highly reliable,
and easily counted--in essence, an accurate representation of ballot
choices whose handling requirements are reasonable. To meet this
objective, there are many factors to consider in an electronic voting
system's design, including:
The environment provided for voting, including the voting
site and various environmental factors,
The ease with which voters can use the voting system,
i.e., its usability,
The robustness and reliability of the voting equipment,
and
The capability of the records to be used in audits.
Independent Dual Verification (IDV) systems have as their primary
objective the production of ballot records that are capable of being
used in audits in which their correctness can be audited to very high
levels of precision. The primary security issues addressed by IDV
systems are:
Whether electronic voting systems are accurately recording
ballot choices, and
Whether the ballot record contents can be audited
precisely post-election.
The threats addressed by IDV systems are those that could cause a
voting system to inaccurately record the voter's intent or cause a
voting system's records to become damaged, i.e., inserted, deleted, or
changed. These threats could occur via any number of means including
accidental damage or various forms of fraud. The threats are addressed
mainly by providing, in the voting system design, the capability for
ballot record audits to detect precisely whether specific records are
correct as recorded or damaged, missing, or fraudulent.

1.1 Independent Dual Verification Systems: Improved Accuracy in Audits

Independent Verification is the top-level categorization for
electronic voting systems that produce multiple records of ballot
choices whose contents are capable of being audited to high levels of
precision. For this to happen, the records must be produced and made
verifiable by the voter, and then subsequently handled according to the
following protocol:
At least two records of the voter's choices are produced
and one of the records is then stored such that it cannot be modified
by the voting system, e.g. the voting system creates a record of the
voter's choices and then copies it to some write-once media.
The voter must be able to verify that both records are
correct, e.g., verify his or her choices on the voting system's display
and also verify the second record of choices stored on the write-once
media.
The verification processes for the two verifications must
be independent of each other and (a) at least one of the records must
be verified directly by the voter, or (b) it is acceptable for the
voter to indirectly verify both records if they are stored on different
systems produced by different vendors.
The content of the two records can be checked later for
consistency through the use of identifiers that allow the records to be
linked.
An assumption is made that at least one set of records is usable in
an efficient counting process such as by using an electronic voting
system, and the other set of records is usable in an efficient process
of verifying its agreement with the other set of records used in the
counting process. The sets of records would preferentially be different
in form and thus have more resistance to accidental or deliberate
damage.
Given these conditions above, the multiple records are said to be
distinct and independently verifiable, that is, both records are not
under the control of the same processes. As a result of this
independence, one record can be used to audit or check up on the
accuracy of the other record. Because the storage of the records is
separate, an attacker who can compromise one of the records still will
face a difficult task in compromising the other.

1.2 Issues in Handling Multiple Records Produced by Independent Dual
Verification Systems

There are several fundamental questions that need to be addressed
when designing the structure and selecting the physical characteristics
of IDV systems records, including:
How to tell if the records are authentic and not forged,
How to tell if the integrity of the records has remained
intact from the time they were recorded,
The suitability of the records for various types of
auditing, and
How best to address problems if there are errors in the
records.
Whenever an electronic voting system produces multiple records of
votes, there is some possibility that one or more of the records may
not match. Records can be lost, or deliberately or accidentally
damaged, or stolen, or fabricated. Keeping the two records in
correspondence with each other can be made more or less difficult
depending on the technologies used for the records and the procedures
used to handle the records.
As a consequence, it is important to structure the records so that
errors and other anomalies can be readily detected during audits. There
are a number of techniques that can be used, such as the following:
Associating unique identifiers with corresponding records,
e.g., an individual paper record sharing a unique identifier with its
corresponding electronic record,
Including an identification of the specific voting system
that produced the records, such as a serial number identifier or by
having the voting system digitally sign the records using public key
cryptography,
Including other information about the election and the
precinct or location where the records were created,
Creating checksums of the electronic records and having
the voting system digitally sign the entire sets of records so that
missing or inserted records can be detected, and
Structuring the records in open, publicly documented
formats that can be readily analyzed on different computing platforms.
The ease or relative difficulty with which some types of records
must be handled is also a determining factor in the practical
capability to conduct precise audits, given that some types of records
are better suited to different types of auditing and different voting
environments than others. The factors that make certain types of
records more suitable than others could vary greatly depending upon
many other criteria, both objective and subjective. For example, paper
records may require manual handling by voters or poll workers and thus
be more susceptible to damage or loss. At the same time, the extent to
which the paper records must be handled will vary depending on the type
of voting system in use. Electronic records may by their nature be more
suitable for automated audits; however electronic records are still
subject to accidental or deliberate damage, loss, and theft.

[[Page 18973]]

2. Core Characteristics for Independent Verification Systems

This section contains a preliminary set of characteristics for IDV
systems. These characteristics are fundamental in nature and apply to
all categories of IDV systems. They will form the basis for future
requirements for independent verification systems.

2.1 An Independent Dual Verification Voting System Produces Two
Distinct Sets of Records of Ballot Choices Via Interactions With the
Voter Such That One Set of Records Can be Compared Against the Other to
Check Their Equality of Content
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is the fundamental core definition for IDV
systems. The records can be checked against one another to determine
whether or not the voter's choices were correctly recorded.
2.1.1 The Voter Verifies the Content of Each Record and Either (a)
Verifies at Least One of the Records Directly or (b) Verifies Both
Records Indirectly if the Records Are Each Under the Control of
Independent Processes
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Direct Verification involves using human senses, e.g.,
directly verifying a paper record via one's eyesight. Indirect
Verification involves using an intermediary to perform the
verification, e.g., verifying an electronic ballot image at the voting
system.
2.1.2 The Creation, Storage, and Handling of the Records are
Sufficiently Separate Such That the Failure or Compromise of One Record
Does Not Cause the Failure or Compromise of Another
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The records must be stored on different media and
handled independently of each other, so that no one process could
compromise all records. If an attack can alter one record, it should
still be very difficult to alter the other record.
2.1.2.1 At Least One Record Is Highly Resistant to Damage or Alteration
and Should be Capable of Long-Term Storage
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: At least one of the records should be difficult to
alter or damage so that it could be used in case the counted records
are damaged or lost.
2.1.3 The Processes of Verification for the Multiple Records do not all
Depend for Their Integrity on the Same Device, Software Module, or
System, and are Sufficiently Separate Such That Each Record Provides
Evidence of the Voter's Choices Independently of Its Other
Corresponding Record
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, the verification of an electronic record
on a DRE is not sufficiently separate from the verification of an
electronic record located on a token but performed by the same DRE as
the verification for the first record. Verification of the paper record
by one's senses is sufficiently separate in this case.
2.1.4 The Records Can Be Used in Checks of One Another, Such That if
One Set of Records Can Be Used in an Efficient Counting Process, the
Other Set of Records Can Be Used for Checking Its Agreement With the
First Set of Records
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, an electronic record can be used in an
efficient counting process. A second paper record can be used to verify
the accuracy of the electronic record; however its suitability for
efficient counting is less clear. If a paper record can be used in an
automated scan process, it may be more suitable.
2.1.5 The Records Within a Set are Linked to Their Corresponding
Records in the Other Set by Including a Unique Identifier Within Each
Record That Can Be Used to Identify the Record's Corresponding Record
in the Other Set
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The identifier should serve the purpose of uniquely
identify the record so as to identify duplicates and/or for cross-
checking two record types.
2.1.6 Each Record Includes an Identification of the Voting Site/
Precinct
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If the voting site and precinct are different, both
should be included.
2.1.7 The Records Include Information Identifying Whether the Balloting
is Provisional, Early, or on Election Day, and Information That
Identifies the Ballot Style In Use
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.1.8 The Records Include a Voting Session Identifier that is Generated
When The Voting Station is Placed in Voting Mode and That Can Be Used
to Identify The Records as Being Created During That Voting Session
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If there are several voting sessions on the same voting
station on the same day, the voting session identifiers must be
different. They should be generated from a random number generator.
2.1.9 The Records Include An Identifier of The Voting System that is
Unique To that Style of Voting Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The identifier could be a serial number or other unique
ID.

[[Page 18974]]

2.1.10 The cryptographic Software in Independent Verification Voting
Systems is Approved by The U.S. Government's Cryptographic Module
Validation Program (CMVP) as Applicable
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voting systems may use cryptographic software for a
number of different purposes, including calculating checksums,
encrypting records, authentication, generating random numbers, and for
digital signatures. This software should be reviewed and approved by
the Cryptographic Module Validation Program. There may be cryptographic
voting schemes where the cryptographic algorithms used are necessarily
different from any algorithms that have approved CMVP implementations,
thus CMVP approved software shall be used where feasible. The CMVP web
site is http://csrc.nist.gov/cryptval.

2. Requirements for Voter Verified Paper Audit Trails (Normative)

This section contains requirements for Voter Verified Paper Audit
Trail (VVPAT) voting systems. VVPAT is not mandatory. These
requirements apply only to voting systems that include a VVPAT
component and are consistent with the definition of Independent Dual
Verification (IDV) systems from Section 6.0.1. Requirements for
usability, accessibility, and privacy from Volume I, Section 2.2.7
apply to VVPAT. The requirements in this section apply only to VVPAT
systems; the requirements do not apply to other types of voting systems
and are not intended to in any way restrict use or operation of other
types of voting systems.

1. Display and Print a Paper Record

1.1 The Voting Station Shall Print and Display a Paper Record of the
Voter's Ballot Choices Prior to the Voter Making the Ballot Choices
Final
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is the basic requirement for VVPAT capability. It
requires that the paper record be created as a distinct representation
of the voter's ballot choices. It requires that the paper record
contain the same information as contained in the electronic record and
be suitable for use in verifications and recounts of the election and
of the voting station's electronic records. Thus, either the paper or
electronic record could be used as the ballot of record for the
election.
1.1.1 The Paper Record Shall Constitute A Complete Record of Ballot
Choices That Can Be Used To Assess The Accuracy of The Voting Station's
Electronic Record, To Verify The Election Results, And In Full
Recounts.
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This requirement exists to make clear that it is
possible to use the paper record for checks of the voting station's
accuracy in recording voter's ballot choices, as well as usable for
election audits (such as mandatory 1% recounts). The paper record shall
also be suitable for use in full manual recounts of the election.
1.1.2 The Paper Record Shall Contain All Information Stored in the
Electronic Record
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The electronic record cannot hide any information
related to ballot choices; all information relating to ballot choices
must be equally present in both records. The electronic record may
contain other items that don't necessarily need to be on the paper
record, such as digital signature information.

2. VVPAT Voting Station Usability

2.1 All Usability Requirements From Volume I, Section 2.2.7 Shall Apply
to Voting Stations With VVPAT
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The requirements in this section are in addition to
those requirements from Section 2.2.7. They require that the paper
record be formatted and displayed so that the voter is able to verify
his or her votes with maximum reasonable ease and satisfaction, and
that instructions be provided to the voter to handle all relevant
aspects of the voter verification.
2.1.1 The Voting Station Shall Be Capable of Showing the Information on
the Paper in a Font Size of at Least 3.0 mm, and Should Be Capable of
Showing the Information in at Least Two Font Ranges, (a) 3.0-4.0 mm and
(b) 6.3-9.0 mm, Under Control of the Voter or Poll Worker
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: In keeping with requirements in Section 2.2.7, the
paper record should use the same font sizes as displayed by the voting
station, but at least be capable of 3.0 mm. While larger font sizes may
assist most voters with poor vision, certain disabilities such as
tunnel vision are best addressed by smaller font sizes.
2.1.2 The Paper and Electronic Records Shall Be Presented so as to
Allow for Easy, Simultaneous Comparison
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.1.2.1 The Paper and Electronic Records Shall Be Positioned so That
the Voter Can, at the Same Posture, Easily Read and Compare the Two
Records
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voter should not have to shift positions when
comparing the records.
2.1.2.2 If The Paper Record Cannot Be Displayed in Its Entirety, a
Means Shall Be Provided to Allow the Voter to View the Entire Ballot
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Possible solutions include scrolling the paper or
printing a new sheet of paper.
2.1.2.3 If the Paper Record Cannot Be Displayed in Its Entirety on a
Single Page, Each Page of the Record Shall Be Numbered and the Last
Page Shall Be Clearly Distinguished
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[[Continued on page 18975]]

From the Federal Register Online via GPO Access [wais.access.gpo.gov]
]

[[pp. 18975-19024]] Technical Guidelines Development Committee (TGDC); Initial
Report: Voluntary Voting System Guidelines Version I

[[Continued from page 18974]]

[[Page 18975]]

2.1.3 The Instructions for Performing the Verification Process Shall Be
Made Available to the Voter in A Location on the Voting Station
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: All instructions need to meet the accessibility
requirements contained in Section 2.2.7.

3. VVPAT Voting Station Accessibility

3.1 All Accessibility Requirements From Section 2.2.7 Shall Apply to
Voting Stations With VVPAT
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Requirements in this section are in addition to the
accessibility and alternative language requirements from Section 2.2.7.
They make explicit that an accessible vote verification procedure for
voters be provided at voting sites, including voters with disabilities,
limited English proficiency (LEP), and voters with Native American and
Alaska Native languages that are not written.
3.1.1 The Voting Station Shall Display, Print, and Store a Paper Record
in any of the Alternative Languages Chosen for Making Ballot Selections
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For the purposes of voter privacy, it must not be
possible to identify voters based on their use of alternative
languages. Requirement 6.0.2.5.1.3 addresses this issue.
3.1.1.1 For the Purposes of Verification, Candidate Names on the
Records Shall Be in English
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This requirement is included to assist manual auditing
of the paper records.
3.1.1.2 Other Markings Not Related to Ballot Selection on The Paper
Record Shall Be In English
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Other markings may include designations of the precinct
and the election.
3.1.2 If the Normal Procedure Includes VVPAT, the Accessible Voting
Station Should Provide Features That Enable Voters Who Are Blind to
Perform This Verification
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This requirement is repeated from Section 2.2.7 and
included here for emphasis. This requirement will be mandatory in
future versions.

4. Approve or Spoil the Paper Record

4.1 The Voting Station Shall Allow the Voter to Approve or Spoil the
Paper Record
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voting station cannot create an electronic record
without its corresponding paper record. It requires that the voting
station mark the electronic record as accepted or spoiled in the
voter's presence, and if spoiled, the corresponding electronic record
be marked as spoiled and be preserved. It requires that the voting
station display a warning message when a spoil limit is reached.
4.1.1 The Voting Station Shall, in the Presence of The Voter, Mark the
Paper Record as Being Accepted by the Voter or Spoiled
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If a paper record is marked as spoiled, then the
corresponding electronic record is presented to the voter for update.
4.1.2 The Voting Station Should Mark and Preserve Electronic and Paper
Records That Have Been Spoiled
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For the purposes of reconciliation of records,
electronic and paper spoiled records should be retained and analyzed.
4.1.3 Following the Close of Polls, a Means Shall Be Provided to
Reconcile the Number of Spoiled Paper Records With the Number of
Occurrences of Spoiled Electronic Records, and Procedures Shall Be in
Place to Address Any Discrepancies
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[Best practice for voting officials] Appropriate procedures are
needed for reconciling the number of spoiled paper records with the
number of spoiled electronic records and for addressing any
discrepancies after the close of polls.
4.1.4 Prior to the Maximum Number of Spoiled Ballots Occurring, the
Voting Station Shall Display a Warning Message to the Voter Indicating
That the Voter May Spoil Only One More Ballot
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The maximum number of spoiled ballots varies from state
to state.
4.1.5 If the Maximum Number of Spoiled Ballots Occurs, the Voting
Station Should Provide a Way to Permit the Voter to Cast a Ballot, as
Required
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Possible solutions include using other equipment, using
a paper ballot, or accepting the last ballot cast. This capability
defined by state and local jurisdiction.
[Best practice for voting officials] Appropriate procedures are
needed to permit the voter to cast a ballot if the maximum number of
spoiled ballots occurs.
[Best practice for voting officials] Appropriate procedures are
needed to address situations in which a voter is unable to review the
paper record.
[Best practice for voting officials] Appropriate procedures are
needed to address situations in which a voter indicates that the
electronic and paper records do not match. If the records do not match,
a potentially serious error has likely occurred, and voting officials
may need to take appropriate actions such as removing the voting
station from service and quarantining its records for later analysis.

[[Page 18976]]

4.1.6 The Voting Station Should Not Record the Electronic Record as
Being Approved By The Voter Until the Paper Record Has Been Stored
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: In general it is better not to record any record as
being approved until the record that is independent of the voting
system is approved by the voter.
4.1.7 Vendor Documentation Shall Include Procedures for Returning a
Voting Station to Correct Operation After a Voter Has Used It
Incompletely or Incorrectly; This Procedure Shall Not Cause
Discrepancies Between the Tallies of the Electronic and Paper Records
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5. Preserve Voter Privacy and Anonymity

5.1 The Voter's Privacy and Anonymity Shall be Preserved During the
Process of Recording, Verifying, and Auditing Ballot Choices
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Privacy requirements from Section 2.2.7 apply to voting
stations with VVPAT; requirements in this section are in addition to
those requirements from Section 2.2.7. They require that the voter's
privacy be maintained during the verification step, including
requirements that the paper record contain no human or machine-readable
markings that could identify the voter and that the paper and
electronic records be stored in ways that preserve the privacy and
anonymity of the voter.
5.1.1 The Privacy and Anonymity of the Voter's Verification of His or
Her Ballot Choices on the Electronic and Paper Records Shall Be
Maintained
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.1.1.1 When the Voter is Responsible for Depositing a Paper Record in
the Ballot Box, the Accessible Voting Station Shall Maintain the
Privacy and Anonymity of Voters Unable to Manually Handle Paper
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.1.2 The Electronic and Paper Records Shall be Created and Stored in
Ways that Preserve the Privacy and Anonymity of the Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This can be accomplished in various ways including
shuffling the order of the records or other methods to separate the
order of stored records.
5.1.3 The Privacy and Anonymity of Voters Whose Paper Records Contain
Any of the Alternative Languages Chosen for Making Ballot Selections
Shall be Maintained
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: One method for accomplishing this is to ensure that no
less than, e.g., five voters use any of the alternative languages for
their ballot selections.
[Best practice for voting officials] Appropriate procedures are
needed to ensure the privacy and anonymity of voters whose paper
records contain any of the alternative languages chosen for making
ballot selections.
5.1.4 The Voter Shall Not be Able to Leave the Voting Area With the
Paper Record if the Information on the Paper Record Can Directly Reveal
the Voter's Choices
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[Best practice for voting officials] Appropriate procedures are
needed to prevent voters from leaving the voting area with a paper
record that can directly reveal the voter's choices.
5.1.5 Unique Identifiers Shall Not be Displayed in a Way That Is Easily
Memorable by the Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Unique identifiers on the paper record are displayed or
formatted in such a way that they are not memorable to voters, such as
by obscuring them in other characters.

6. Electronic and Paper Record Structure

6.1 The Voting Station's Ballot Records Shall Be Structured and Contain
Information So as to Support Highly Precise Audits of Their Accuracy
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: It requires that electronic records and paper records
contain election precinct information, information to link the paper
record to its corresponding electronic record, and information
identifying the voting station. It requires that the electronic records
be maintained in a format that can be exported to a different computer,
e.g., a personal computer, and that the format be well-documented to
support analysis of the records.
6.1.1 All Cryptographic Software in the Voting Station Should be
Approved by the U.S. Government's Cryptographic Module Validation
Program (CMVP) as Applicable
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voting station may use cryptographic software for a
number of different purposes, including calculating checksums,
encrypting records, authentication, generating random numbers, and for
digital signatures. This software should be reviewed and approved by
the Cryptographic Module Validation Program. There may be cryptographic
voting schemes where the cryptographic algorithms used are necessarily
different from any algorithms that have approved CMVP implementations,
thus CMVP approved software should be used where feasible but is not
required. The CMVP web site is http://csrc.nist.gov/cryptval.

[[Page 18977]]

6.1.2 The Electronic and Paper Records Shall Include Information About
the Election
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

6.1.2.1 The Voting Station Shall be Able to Include an Identification
of the Particular Election, the Voting Site/Precinct, and the Voting
Station
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If the voting site and precinct are different, both
should be included. Some of this information may have to be excluded in
certain cases to protect voter privacy.
6.1.2.2 The Records Shall Include Information Identifying Whether the
Balloting is Provisional, Early, or on Election Day, and Information
that Identifies the Ballot Style in Use
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

6.1.2.3 The Records Shall Include a Voting Session Identifier That is
Generated When the Voting Station is Placed in Voting Mode and That can
be Used to Identify the Records as Being Created During that Voting
Session
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If there are several voting sessions on the same voting
station on the same day, the voting session identifiers must be
different. They should be generated from a random number generator.
6.1.3 The Electronic and Paper Records Shall be Linked by Including a
Unique Identifier Within Each Record That can be Used to Identify Each
Record Uniquely and Each Record's Corresponding Record
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The identifier serves the purpose of uniquely
identifying the record so as to identify duplicates and/or for
crosschecking two record types.
6.1.4 The Voting Station Should Generate and Store a Digital Signature
for Each Electronic Record
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

6.1.5 The Electronic Records Shall be Able to be Exported for Auditing
or Analysis on Standards Based and/or COTS Information Technology
Computing Platforms
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

6.1.5.1 The Exported Electronic Records Shall be in a Publicly
Available, Non-Proprietary Format
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: It is advantageous when all electronic records,
regardless of manufacture, use the same format or can easily be
converted to a publicly available, non-proprietary format, e.g., the
OASIS Election Markup Language (EML) Standard.
6.1.5.2 The Voting Station Should Export the Records Accompanied by a
Digital Signature of the Collection of Records, Which Shall be
Calculated on the Entire Set of Electronic Records and Their Associated
Digital Signatures
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is necessary to determine if records are missing
or substituted.
6.1.5.3 The Voting System Vendor Shall Provide Documentation as to the
Structure of the Exported Records and How They Shall be Read and
Processed by Software
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

6.1.5.4 The Voting System Vendor Shall Provide a Software Program That
Will Display the Exported Records and That May Include Other
Capabilities Such as Providing Vote Tallies and Indications of
Undervotes
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

6.1.6 The Paper Record Should be Created in a Format That May be Made
Available Across Different Manufacturers of Electronic Voting Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Future standards may require some commonality in the
format of paper records.
6.1.7 The Paper Record Shall be Created Such That Its Contents Are
Machine-Readable
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This can be done by using specific OCR fonts.
6.1.7.1 The Paper Record Should Contain Error Correcting Codes for the
Purposes of Detecting Read Errors and for Preventing Other Markings on
the Paper Record to be Misinterpreted When Machine Reading the Paper
Record
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This requirement is not mandatory if, for example, a
state prohibits non-human-readable information on the paper record.
This requirement serves the purpose of detecting scanning errors and
preventing stray or deliberate markings on the paper from being
interpreted as valid data.
6.1.8 Any Automatic Accumulation of Electronic or Paper Records Shall
be Capable of Detecting and Discarding Duplicate Copies of the Records
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[[Page 18978]]

6.1.9 The Voting Station Should be Able to Print a Barcode With Each
Paper Record that Contain the Human Readable Contents of the Paper
Record and Digital Signature Information
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This requirement is not mandatory if, for example, a
state prohibits non-human-readable information on the paper record.
6.1.9.1 The Barcode Shall Use an Industry-Standard Format and Shall be
Able to be Read Using Readily Available Commercial Technology
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Examples of such codes are Maxi Code or PDF417.
6.1.9.2 If the Paper Record's Corresponding Electronic Record Contains
a Digital Signature, the Digital Signature Shall be Included in the
Barcode
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

6.1.9.3 The Barcode Shall Not Contain Any Information Other Than the
Paper Record's Human Readable Content and Digital Signature Information
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

6.1.10 The Voting System Vendor Shall Provide Full Documentation of
Procedures for Exporting Its Electronic Records and Reconciling Its
Electronic Records With Its Paper Records
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7. Equipment Security and Reliability

7.1 The Voting Station Equipment Shall be Secure, Reliable, and Easily
Maintained
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.1 The Voting Station Shall be Physically Secure From Tampering,
Including Intentional Damage
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[Best practice for voting officials] Appropriate procedures are
needed to ensure that voting systems are physically secured from
tampering and intentional damage.
7.1.1.1 The Voting Station Shall Provide a Standard, Publicly
Documented Printer Port (or the Equivalent) Using a Standard
Communication Protocol
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Using a standard, publicly documented printer protocol
assists in security evaluations of its software.
7.1.1.2 The Paper Path Between the Printing, Viewing and Storage of The
Paper Record Shall be Protected and Sealed From Access Except by
Authorized Election Officials
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.1.3 The Printer Shall Not be Permitted to Communicate With Any
Other System or Machine Other Than the Single Voting Machine to Which
it is Connected
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.1.4 The Printer Shall Only be Able to Function as a Printer; It
Shall Not Contain Any Other Services (e.g., Provide Copier or Fax
Functions) or Network Capability
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.1.5 Printer Access to Replace Consumables Such as Ink or Paper
Shall Only be Possible if it Does Not Compromise the Sealed Printer
Paper Path
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.1.6 The Ballot Box Storing the Paper Records Shall be Sealed and
Secured and No Access Shall be Provided to Poll Workers
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.1.7 Tamper-Evident Seals or Physical Security Measures Shall
Protect the Connection Between the Printer and the Voting Station, so
that the Connection Cannot be Broken or Interfered With Without Leaving
Extensive and Obvious Evidence
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.2 The Voting Station's Printer Shall be Highly Reliable and Easily
Maintained
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.2.1 The Voting Station Should Detect Errors and Malfunctions such
as Paper Jams or Low Supplies of Consumables such as Paper and Ink That
May Prevent Paper Records from Being Correctly Displayed Printed or
Stored
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This could be accomplished in a variety of different
ways: for example, a printer that is out of paper or jammed could issue
audible alarms, with the alarm different for each condition.

[[Page 18979]]

7.1.2.2 If Errors or Malfunctions Occur, the Voting Station Shall
Suspend Voting Operations and Should Present a Clear Indication to The
Voter and Election Officials of the Malfunctions
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voting station does not record votes if errors or
malfunctions occur.
7.1.2.3 Printing Devices Should Either (a) Contain Paper and Ink of
Sufficient Capacity so as not to Require Reloading or Opening Equipment
Covers or Enclosures and Circumvention of Security Features, or (b) Be
Able to Reload Paper and Ink with Minimal Disruption to Voting and
Without Circumvention of Security Features such as Seals
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.2.4 Vendor Documentation Shall Include Procedures for Investigating
and Resolving Printer Malfunctions Including but Not Limited to Printer
Operations, Misreporting of Votes, Unreadable Paper Records, and Power
Failures
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.2.5 Vendor Documentation Shall Include Printer Reliability
Information Including Mean Time Between Failure Information and Shall
Include Recommendations for Appropriate Numbers of Backup Printer and
Printer Supplies
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.3 Protective Coverings Intended to be Transparent on Voting Station
Devices Shall be Maintainable via a Predefined Cleaning Process. If The
Coverings Become Damaged Such That They Obscure the Paper Record, They
Shall be Replaceable
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

7.1.4 The Paper Record Shall be Sturdy, Clean, and of Sufficient
Durability to be Used for Verifications, Reconciliations, and Recounts
Conducted Manually and via Machine Reading Equipment
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3. Wireless Requirements (Normative)

This section provides wireless requirements for implementing and
using wireless capabilities within a voting system. These requirements
reduce, but don't eliminate, the risk of using wireless communications
for voting systems.
Wireless is defined as any means of communication that occurs
without wires. This normally covers the entire electromagnetic
spectrum. For the purposes of this section wireless includes radio
frequency (RF), infrared, (IR), and microwave.
Since the wireless communications path on which the signals travel
is via the air and not via a wire or cable, devices other than those
intended to receive the wireless signal (e.g., voting data) can receive
(intentionally and unintentionally) the wireless signals. Some of the
wireless communications paths (i.e., signals) are weakened by walls and
distance, but are not stopped. This makes it possible to eavesdrop from
a distance as well as transmit wireless signals (e.g., interference or
intrusive data) from a distance. In many cases the wireless signals
cannot be seen, heard, or felt, thus making the presence of wireless
communication hard to determine by the human senses. The use of
wireless technology introduces severe risk and should be approached
with extreme caution. The requirements in this section (i.e.,
controlling and identifying usage, protecting the transmitted data and
path, and protecting the system) mitigate these risks.
The requirements that are applicable to all types of wireless
communications are presented, followed by requirements that are
applicable to a specific part of the electromagnetic spectrum (e.g.,
audible, radio frequency, and infrared). These latter requirements only
apply to systems using those parts of the spectrum.
There are other concerns when evaluating wireless usage,
specifically radio frequency. A device's radio frequencies usage and
the power output are governed by Federal Communications Commission
(FCC) regulations and therefore all RF wireless communications devices
are subject to the applicable FCC requirements. However, these FCC
regulations do not fully address RF wireless interference caused by
multiple FCC compliant devices. That is, the RF wireless used in a
voting system may be using the same RF wireless of another non-voting
wireless system and which may potentially cause a degradation of the
wireless performance or a complete wireless failure for the voting
system. Sometimes a particular wireless technology permits a power
output range, which may be used to overcome interference received from
another device. A radio emissions site test can determine the extent of
potential existing interference at the location where the wireless
voting system is to be used. A radio emission site test can also
determine the extent that the RF wireless transmission of the voting
system escapes the building in which the RF wireless voting system is
used.

1. Relationship to Volume I, Section 5: ``Telecommunications''

1.1 At a Minimum Wireless Communications Shall Meet the Requirements
Listed in Volume I, Section 5, ``Telecommunications''
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

2. Controlling Usage

2.1 If Wireless Communications Are Used in a Voting System, Then the
Vendor Shall Supply Documentation Describing How to Use All Aspects of
Wireless Communications in a Secure Manner
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

2.1.1 This documentation shall include:
A complete description of the uses of wireless in the
voting system including descriptions of the data elements and signals
that are to be carried by the wireless mechanism,
A complete description of the vulnerabilities associated
with this proposed use of wireless, including vulnerabilities deriving
from the insertion, deletion, modification, capture, or suppression of
wireless messages,
A complete description of the techniques used to mitigate
the risks associated with the described

[[Page 18980]]

vulnerabilities including techniques used by the vendor to ensure that
wireless cannot send or receive messages other than those situations
specified in the documentation. Cryptographic techniques shall be
carefully and fully described, including a description of cryptographic
key generation, management, use, certification, and destruction, and
A rationale for the inclusion of wireless in the proposed
voting system, based on a careful and complete description of the
perceived advantages and disadvantages of using wireless for the
documented uses compared to using non-wireless approaches.
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

Discussion: In general, convenience is not a sufficiently
compelling reason, on its own, to justify the inclusion of wireless
communications in a voting system. If convenience is cited as an
advantage of wireless, it shall be balanced against the difficulty of
working with cryptographic keys.
[Best Practice for Voting Officials] When using encryption to
ensure that the wireless communication is secure, appropriate
procedures are needed for cryptographic key management.
2.1.2 The Details of All Cryptographic Protocols Used for Wireless
Communications, Including the Specific Features and Data, Shall be
Documented
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

2.1.3 The Wireless Documentation Shall be Closely Reviewed for
Accuracy, Completeness, and Correctness
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

2.1.3.1 This Review Shall be Done Either Through an Open and Public
Review or by a Subject Area Recognized Expert
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

2.1.4 There Shall be No Undocumented Use of the Wireless Capability,
Nor Shall There be Any Use of the Wireless Capability That Is Not
Entirely Controlled by the Voting Official
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

Discussion: This can be tested by reviewing all of the software,
hardware, and documentation and by testing the status of wireless
activity during all phases of testing.

2.2 If a Voting System Includes Wireless Capabilities, Then the Voting
System Should be Able to Accomplish the Same Function if Wireless
Capabilities are Not Available Due to an Error or No Service
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

2.2.1 The Vendor Shall Provide Documentation How to Accomplish These
Functions When Wireless Is Not Available
[GRAPHIC] [TIFF OMITTED] TN12AP06.005

2.3 The System Shall be Designed and Configured Such That it Is Not
Vulnerable to a Single Point of Failure Using Wireless Communications
That Causes a Total Loss of Any of Voting Capabilities
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

Discussion: Rewritten from Volume 1, Section 5.2.6 Integrity item
(c)

2.4 If a Voting System Includes Wireless Capabilities, Then the System
Shall Have the Ability to Turn on the Wireless Capability When it is to
be Used and to Turn Off the Wireless Capability When the Wireless
Capability is Not in Use
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

2.5 If a Voting System Includes Wireless Capabilities, Then the System
Shall Not Activate the Wireless Capabilities without Confirmation From
a Voting Official
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

3. Identifying Usage

Since there are a wide variety of wireless technologies (both
standard and proprietary) and differing physical properties of wireless
signals, it is important to identify some of the characteristics of the
wireless technologies used in the voting system

3.1 If a Voting System Provides Wireless Communications Capabilities,
Then There Shall be a Method for Determining the Existence of the
Wireless Communications Capabilities
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

3.2 If a Voting System Provides Wireless Communications Capabilities,
Then There Shall be An Indication that Allows One to Determine When the
Wireless Communications (e.g., Radio Frequencies) Capability is Active
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

3.2.1 The Indication Should be Visual
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

3.3 If a Voting System Provides Wireless Communications Capabilities,
Then the Type of Wireless Communications Used (e.g., Radio Frequencies)
Shall be Identified Either via a Label or via the Voting Systems
Documentation
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

4. Protecting the Transmitted Data

The transmitted data, especially via wireless communications, needs
to be protected to ensure confidentiality and integrity. Examples of
election information that needs to be protected

[[Page 18981]]

include: ballot definitions, ballot instructions (audio), voting device
counts, precinct counts, opening of poll signal, and closing of poll
signal.
Examples of non--specific election information that needs to be
protected include: Protocol messages, address or device identification
information, and passwords.
Since radio frequency wireless signals radiate in all directions
and pass through most construction material, anyone may easily receive
the wireless signals. In contrast, infrared signals are line of sight
and do not pass through most construction materials. However to a
lesser extent, infrared signals can still be received by other devices
that are in the line of sight. Similarly, wireless signals can also be
easily transmitted by others in order to create unwanted signals. Thus
to protect the privacy and confidentiality of the information,
encryption is required. The following requirements are rewritten from
Volume I, Section 6.5.3.

4.1 All Information Transmitted via Wireless Communications Shall be
Encrypted and Authenticated, with the Exception of Wireless T-Coil
Coupling, to Protect Against Eavesdropping and Data Manipulation
Including Modification, Insertion, and Deletion
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

4.1.1 The Encryption Shall be as Defined in Federal Information
Processing Standards (FIPS) 197, ``Advanced Encryption Standard (AES)''
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

4.1.1.1 The Cryptographic Modules Used Shall Comply With FIPS 140-2,
Security Requirements for Cryptographic Modules
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

4.1.2 The Capability to Transmit Non-Encrypted and Non-Authenticated
Information via Wireless Communications Shall Not Exist
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

4.1.2.1 If Wireless Communication (Audible) is Used, and if the
Receiver of the Wireless Transmission is the Human Ear, then the
Information Shall Not be Encrypted (i.e., This Specifically Covers the
Case of the Wireless T-Coil Coupling for Assistive Devices Used by
People Who are Hard of Hearing--See Volume I, Section 2.2.7.2 DRE
Standards Item C)
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

5. Protecting the Wireless Path

With the exception of wireless communications using audible and
infrared, it is technically infeasible to use physical means to prevent
denial of service (DoS) attacks. If wireless communications are used,
then the following capabilities shall exist in order to mitigate the
effects of a denial of service (DoS) attack:

5.1 The Voting System Shall be Able to Function Properly Throughout a
DoS Attack, Since the DoS Attack May Continue Throughout the Voting
Process.
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

5.2 The Voting System Shall Function Properly as if the Wireless
Capability Were Never Available for Use
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

5.3 Alternative Procedures or Capabilities Shall Exist to Accomplish
the Same Functions That the Wireless Communications Capability Would
Have Done
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

5.4 The Wireless (Audible) Path Shall be Protected or Shielded
[GRAPHIC] [TIFF OMITTED] TN12AP06.010

Discussion: Protecting the audible path is a tradeoff between the
high volume level necessary for an individual to hear with the low
volume level necessary to keep others from hearing, as well as
protecting from interference (i.e., noise) from the polling place,
voting station, or voting environment. The same is true for the audible
path if a voter's speech is to be captured by the voting device. This
wireless communication's path protection is necessary to protect
privacy. Some audio headsets may already satisfy this requirement for
the hearing part, while a soundproof voting booth may be necessary in
some other cases (e.g., voice recordings).

5.5 Infrared

Since infrared has the line-of-sight (LoS) property, securing the
wireless path can be accomplished by shielding the path between the
wireless communicating devices with an opaque enclosure. However this
is only practical for short distances. Additionally, this type of
shielding can help to prevent accidental damage to the eyes by the
infrared signal.
5.5.1 The Shielding Shall be Strong Enough to Prevent Escape of the
Voting System's Signal, as well as Strong Enough to Prevent Infrared
Saturation Jamming
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

6. Protecting the Voting System from a Wireless-based Attack

The security of the wireless voting systems is as important as the
information transmitted. If a voting system becomes compromised, there
is no way to determine the harm to the system until the compromise is
discovered and an investigation is conducted to determine the extent of
the damage.
Physical security measures (Volume I, Section 6.3) to prohibit
access to a voting system are not possible when using a wireless
communications interface. This is similar to when access is through a
telecommunications interface, but it is worsened by the fact that there
is no wire (physical communication path) to physically secure and by
the various physical properties of the electromagnetic spectrum used.
This section covers and reaffirms the applicable overall system
capabilities defined in Volume I, Section 2 as well as authentication
requirements.

[[Page 18982]]

6.1 The Security Requirements Listed in Volume I, Section 2.2.1 Shall
be Applicable to Systems With Wireless Communications
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

6.2 The Accuracy Requirements Listed in Volume I, Section 2.2.2 Shall
be Applicable to Systems With Wireless Communications
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

6.2.1 The Use of Wireless Communications That May Cause Impact to the
System's Accuracy Through Electromagnetic Stresses Is Prohibited
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

6.3 The Error Recovery Requirements Listed in Volume I, Section 2.2.3,
Shall Be Applicable to Systems With Wireless Communications
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

6.4 All Wireless Communications Actions Shall Be Logged
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

Discussion: A log of important information is maintained to monitor
the wireless communications. This is to ensure that the wireless
communications are only used by authorized users with authorized access
to authorized devices or services, or to determine if these
requirements were not followed. This relates to the system audit
requirements (Volume I, Section 2.2.5) and integrity (Volume I, Section
2.2.4), if wireless communications are used.
6.4.1 The Log Shall Contain at Least the Following Entries: Times
Wireless Activated and Deactivated, Services Accessed, Identification
of Device to Which Data Was Transmitted to or Received From,
Identification of Authorized User, and Successful and Unsuccessful
Attempts To Access Wireless Communications or Service
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

Discussion: Other information such as the number of frames or
packets transmitted or received at various logical layers may be
useful, but is dependent on the wireless technology used.
[Best Practice for Voting Officials] Appropriate procedures are
needed to ensure that wireless communication actions are logged and
capture at least the following information: Times wireless activated
and deactivated, services accessed, identification of device to which
data was transmitted to or received from, identification of authorized
user, and successful and unsuccessful attempts to access wireless
communications or service.

6.5 Authentication

Authentication is an important part in the protection and security
of the wireless communications. It provides a mechanism to verify the
identity and legitimacy of a person, device, services, or system.
Authenticating users, devices and services helps to secure the wireless
communications and prevent unauthorized access to the system, services
and/or information.
6.5.1 Device Authentication Shall Occur Before Any Access to or
Services From the Voting System are Granted Through Wireless
Communications
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

6.5.2 User Authentication Shall Be at Least Level 2 as Per NIST Special
Publication 800-63 Version 1.0.1, ``Electronic Authentication
Guideline.''
[GRAPHIC] [TIFF OMITTED] TN12AP06.006

4. Distribution of Voting System Software and Setup Validation
(Normative)

This section specifies requirements for the distribution of voting
system software and the setup validation performed on voting system
equipment. These requirements are applicable to voting systems that
have completed qualification testing. The goal of the software
distribution requirements is to ensure that the correct voting system
software has been distributed without modification. The goal of setup
validation requirements, including requirements for verifying the
presence of qualified software and the absence of other software, is to
ensure that voting system equipment is in a proper initial state before
being used.
In general, a voting system can be considered to be composed of
multiple other systems including polling place systems, central
counting/aggregation systems, and election management systems. These
other systems may reside on different computer based platforms at
different locations and run different software. Voting system software
is considered to be all executable code and associated configuration
files critical for the proper operation of the voting system regardless
of the location of installation and functionality provided. This
includes third party software such as operating systems, drivers, etc.

1. Software Distribution Methodology Requirements

1.1 The Vendor Shall Document All Software Including Voting System
Software, Third Party Software (Such as Operating Systems, Drivers,
etc.) To Be Installed on Voting Equipment of the Qualified Voting
System, and Installation Programs
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

1.1.1 The Documentation Shall Have a Unique Identifier (Such as a
Serial Number) for the Following Set of Information: Documentation,
Software Vendor Name, Product Name, Version, Qualification Number of
the Voting System, File Names and Paths or Other Location Information
(Such as Storage Addresses) of the Software
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

[[Page 18983]]

1.1.2 The Documentation Shall Designate All Software Files as Static,
Semi-Static, or Dynamic
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

Discussion: Static voting system software such as executable code
does not change based on the election being conducted or the voting
equipment upon which it is installed. Semi-static voting system
software contains configuration information for the voting system based
on the voting equipment that is installed and the election being
conducted. Semi-static software is only modified during the
installation of (a) the voting system software on voting equipment or
(b) the election specific software such as ballot formats. Dynamic
voting system software changes over time once installed on voting
equipment. However, the specific time or value of the change in the
dynamic software is usually unknown a priori making it impossible to
create reference information to verify the software.

1.2 The EAC Accredited Testing Authority Shall Witness the Final Build
of the Executable Version of the Qualified Voting System Software
Performed by the Vendor
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

1.2.1 The Testing Authority Shall Create a Complete Record of the Build
That Includes: A Unique Identifier (Such as a Serial Number) for the
Complete Record, List of Unique Identifiers of Write-Once Media
Associated with the record, Time, Date, Location, Name and Signatures
of All People Present, Source Code and Resulting Executable File Names,
Version of Voting System Software, Qualification Number of the Voting
System, the Name and Versions of All (Including Third Party) Libraries,
and the Name, Version, and Configuration Files of the Development
Environment Used for the Build
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

1.2.2 The Record of the Source Code and Executable Files Shall be Made
on Write-Once Media. Each Piece of Write-Once Media Shall Have a Unique
Identifier
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

Discussion: Write-once media includes technology such as a CD-R,
ROM, or PROM (but not EEPROM or CD-RW). The unique identifiers appear
on indelibly printed labels and in a digitally signed file on the
write-once media.
1.2.3 The Testing Authority Shall Retain This Record Until the Voting
System Ceases to be Qualified
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

1.2.4 The EAC Accredited Testing Authority Shall Create a Subset of the
Complete Record of the Build That Includes a Unique Identifier (Such as
a Serial Number) of the Subset, the Unique Identifier of the Complete
Record, List of Unique Identifiers of Write-Once Media Associated With
the Subset, Vendor, Product Name, Version of Voting System Software,
Qualification Number of the Voting System, All the Files That Resulted
from the Build and Binary Images of All Installation Programs
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

1.2.5 The Record of the Software Shall be Made on Write-Once Media.
Each Piece of Write-Once Media Shall Have a Unique Identifier
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

1.2.6 The Testing Authority Shall Retain a Copy, Send a Copy to the
Vendor, and Send a Copy to the NIST National Software Reference Library
(NSRL) \1\ and/or to Any Other Repository Named by the Election
Assistance Commission
---------------------------------------------------------------------------

\1\ The National Software Reference Library (NSRL) is a
repository of software established and directed by the National
Institute of Standards and Technology. It was designed to meet the
need for court admissible evidence in the identification of software
files. The EAC designated the NSRL as a repository for voting system
software.
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

Discussion: The NSRL was established to meet the needs of the law
enforcement community for court admissible digital evidence by
providing an authoritative source of commercial software reference
information. Information is available at http://www.nsrl.nist.gov.

1.2.7 The Testing Authority Shall Retain This Record Until the Voting
System Ceases to be Qualified
[GRAPHIC] [TIFF OMITTED] TN12AP06.007

1.3 The Vendor Shall Provide the NSRL or Other EAC Designated
Repository With a Copy of All Third Party Software
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

1.4 All Voting System Software, Installation Programs, Third Party
Software (Such as Operating Systems, Drivers, etc.) Used to Install or
to be Installed on Voting System Equipment Shall be Distributed on a
Write-Once Media
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

[Best Practice for Voting Officials] Voting software used to
install the qualified voting systems can be obtained on write-once
media from the voting system vendor or an EAC accredited testing
authority.

[[Page 18984]]

1.4.1 The Vendor Shall Document That the Process Used to Verify the
Software Distributed on Write-Once Media is the Qualified Software by
Using the Reference Information Provided by the NSRL or Other EAC
Designated Repository
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

[Best Practice for Voting Officials] The reference information
produced by the NSRL or other EAC designated repository can be used to
verify that the correct software has been received.
1.4.2 The Voting System Equipment Shall be Designed to Allow the Voting
System Administrator to Verify That the Software is the Qualified
Software by Comparing it to Reference Information Produced by the NSRL
or Other EAC Designated Repository Before Installing the Software
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

1.4.3 The Vendors and Testing Authority Shall Document to Whom They
Provide Voting System Software Write-Once Media
[GRAPHIC] [TIFF OMITTED] TN12AP06.004

2. Generation and Distribution Requirements for Reference Information

2.1 The NSRL or Other EAC Designed Repository Shall Generate Reference
Information Using the Binary Images of the (a) Qualified Voting System
Software Received on Write-Once Media From Testing Authorities and (b)
Election Specific Software Received on Write-Once Media From
Jurisdictions
[GRAPHIC] [TIFF OMITTED] TN12AP06.009

2.1.1 The NSRL or Other EAC Designated Repository Shall Generate
Reference Information in at Least One of the Following Forms: (a)
Complete Binary Images, (b) Cryptographic Hash Values, or (c) Digital
Signatures of the Software
[GRAPHIC] [TIFF OMITTED] TN12AP06.009

Discussion: Although binary images, cryptographic hashes, and
digital signatures can detect a modification or alteration in the
software, they cannot determine if the change to the software was
accidental or intentional.
2.1.1.1 The NSRL or Other EAC Designated Repository Shall Create a
Record of the Creation of Reference Information That Includes: A Unique
Identifier (Such as a Serial Number) for the Record, File Names of
Software and Associated Unique Identifier(s) of the Write-Once Media
From Which Reference Information is Generated, Time, Date, Name of
People Who Generated Reference Information, the Type of Reference
Information Created, Qualification Number of Voting System (If Issued),
Voting System Software Version, Product Name, and Vendor
[GRAPHIC] [TIFF OMITTED] TN12AP06.009

2.1.1.2 The NSRL or Other EAC Designated Repository Shall Retain the
Write-Once Media Used to Generate the Reference Information Until the
Voting System Ceases to be Qualified
[GRAPHIC] [TIFF OMITTED] TN12AP06.009

2.1.1.3 The NSRL or Other EAC Designated Repository That Generates Hash
Value and/or Digital Signature Reference Information Shall Use FIPS
Approved Algorithms for Hashing and Signing
[GRAPHIC] [TIFF OMITTED] TN12AP06.009

2.1.1.4 The NSRL or Other EAC Designated Repository That Generates Hash
Values, Digital Signatures Reference Information, or Cryptographic Keys
Shall Use a FIPS 140-2 Level 1 or Higher Validated Cryptographic Module
[GRAPHIC] [TIFF OMITTED] TN12AP06.009

Discussion: See http://www.csrc.nist.gov/cryptval/ for information

on FIPS 140-2.
2.1.1.5 The NSRL or Other EAC Designated Repository That Generates Sets
of Hash Values and Digital Signatures for Reference Information Shall
Include a Hash Value or Digital Signature Covering the Set of Reference
Information
[GRAPHIC] [TIFF OMITTED] TN12AP06.009

6.1 Scope

This section describes essential security capabilities for a voting
system, encompassing the system's hardware, software, communications,
and documentation. The Standards recognize that no predefined set of
security standards will address and defeat all conceivable or
theoretical threats. However, the Standards articulate requirements to
achieve acceptable levels of integrity, reliability, and inviolability.
Ultimately, the objectives of the security standards for voting systems
are:
To establish and maintain controls that can ensure that
accidents, inadvertent mistakes, and errors are minimized,
To protect the system from intentional manipulation and
fraud, and from malicious mischief,
To identify fraudulent or erroneous changes to the system,
and
To protect secrecy in the voting process.
The Standards are intended to address a broad range of risks to the
integrity of a voting system. While it is not possible to identify all
potential risks, the Standards identify several types of risk that must
be addressed by a voting system. These include:
Unauthorized changes to system capabilities for:
Defining ballot formats,
Casting and recording votes,
Calculating vote totals consistent with defined ballot
formats, and
Reporting vote totals,

[[Page 18985]]

Alteration of voting system audit trails,
Changing, or preventing the recording of, a vote,
Introducing data for a vote not cast by a registered
voter,
Changing calculated vote totals,
Preventing access to vote data, including individual votes
and vote totals, to unauthorized individuals, and
Preventing access to voter identification data and data
for votes cast by the voter such that an individual can determine the
content of specific votes cast by the voter.
This section describes specific capabilities that vendors shall
integrate into a voting system in order to address the risks listed
above.
6.1.1 System Components and Sources
The requirements of this section apply to the broad range of
hardware, software, communications components, and documentation that
comprises a voting system. These requirements apply to components:
Provided by the voting system vendor and the vendor's
suppliers,
Furnished by an external provider (for example providers
of personal computers and commercial off-the-shelf (COTS) operating
systems) where the components are capable of being used during voting
system operation, and
Developed by a voting jurisdiction.
6.1.2 Location and Control of Software and Hardware on Which It
Operates
The requirements of this section apply to all software used in any
manner to support any voting-related activity, regardless of the
ownership of the software or the ownership and location of the hardware
on which the software is installed or operated. These requirements
apply to software that operates on:
Voting devices and vote counting devices installed at
polling places under the control or authority of the voting
jurisdiction, and
Ballot printers, vote counting devices, and other hardware
typically installed at central or precinct locations (including
contractor facilities).
However, some requirements are applicable only in circumstances
specified by this section.
6.1.3 Elements of Security Outside Vendor Control
The requirements of this section apply to the capabilities of a
voting system provided by the vendor. The Standards recognizes that
effective security requires safeguards beyond those provided by the
vendor. Effective security demands diligent security practices by the
purchasing jurisdiction and the jurisdictions representatives. These
practices include:
Administrative and management controls for the voting
system and election management, including access controls,
Internal security procedures,
Adherence to, and enforcement of, operational procedures
(e.g., effective password management),
Security of physical facilities, and
Organizational responsibilities and personnel screening.
Because specific standards for these elements are not under the
direct control of the vendor, they will be addressed in forthcoming
Operational Guidelines that address best practices for jurisdictions
conducting elections and managing the operation of voting systems.
6.1.4 Organization of this Section
The standards presented in this section are organized as follows:
Access Control: These standards addresses procedures and
system capabilities that limit or detect access to critical system
components in order to guard against loss of system integrity,
availability, confidentiality, and accountability.
Equipment and Data Security: These standards address
physical security measures and procedures that prevent disruption of
the voting process at the poll site and corruption of voting data.
Software Security: These standards address the
installation of software, including firmware, in the voting system and
the protection against malicious software.
Telecommunication and Data Transmission: These standards
address security for the electronic transmission of data between system
components or locations over both private and public networks
Security for Transmission of Official Data Over Public
Communications Networks: These standards address security for systems
that communicate individual votes or vote totals over public
communications networks.
It should be noted that computer-generated audit controls
facilitate system security and are an integral part of software
capability. These audit requirements are presented in section 4.

6.2 Access Control

Access controls are procedures and system capabilities that detect
or limit access to system components in order to guard against loss of
system integrity, availability, confidentiality, and accountability.
Access controls provide reasonable assurance that system resources such
as data files, application programs, and computer-related facilities
and equipment are protected against unauthorized operation,
modification, disclosure, loss, or impairment. Unauthorized operations
include modification of compiled or interpreted code, run-time
alteration of flow control logic or of data, and abstraction of raw or
processed voting data in any form other than a standard output report
by an authorized operator.
Access controls may include physical controls, such as keeping
computers in locked rooms to limit physical access, and technical
controls, such as security software programs designed to prevent or
detect unauthorized access to sensitive files. The access controls
contained in this section of the Standards are limited to those
controls required of system vendors. Access controls required of
jurisdictions will be addressed in future documents detailing
operational guidelines for jurisdictions.
6.2.1 Access Control Policy
The vendor shall specify the general features and capabilities of
the access control policy recommended to provide effective voting
system security.
6.2.1.1 General Access Control Policy
Although the jurisdiction in which the voting system is operated is
responsible for determining the access policies applying to each
election, the vendor shall provide a description of recommended
policies for:
a. Software access controls,
b. Hardware access controls,
c. Communications,
d. Effective password management,
e. Protection abilities of a particular operating system,
f. General characteristics of supervisory access privileges,
g. Segregation of duties, and
h. Any additional relevant characteristics.
6.2.1.2 Individual Access Privileges
Voting system vendors shall:
a. Identify each person to whom access is granted, and the specific
functions and data to which each person holds authorized access,
b. Specify whether an individual's authorization is limited to a
specific time, time interval, or phase of the voting or counting
operations, and
c. Permit the voter to cast a ballot expeditiously, but preclude
voter access to all other aspects of the vote-counting processes.

[[Page 18986]]

6.2.2 Access Control Measures
Vendors shall provide a detailed description of all system access
control measures designed to permit authorized access to the system and
prevent unauthorized access. Examples of such measures include:
a. Use of data and user authorization,
b. Program unit ownership and other regional boundaries,
c. One-end or two-end port protection devices,
d. Security kernels,
e. Computer-generated password keys,
f. Special protocols,
g. Message encryption, and
h. Controlled access security.
Vendors also shall define and provide a detailed description of the
methods used to prevent unauthorized access to the access control
capabilities of the system itself.

6.3 Physical Security Measures

A voting system's sensitivity to disruption or corruption of data
depends, in part, on the physical location of equipment and data media,
and on the establishment of secure telecommunications among various
locations. Most often, the disruption of voting and vote counting
results from a physical violation of one or more areas of the system
thought to be protected. Therefore, security procedures shall address
physical threats and the corresponding means to defeat them.
6.3.1 Polling Place Security
For polling place operations, vendors shall develop and provide
detailed documentation of measures to anticipate and counteract
vandalism, civil disobedience, and similar occurrences. The measures
shall:
a. Allow the immediate detection of tampering with vote casting
devices and precinct ballot counters, and
b. Control physical access to a telecommunications link if such a
link is used.
6.3.2 Central Count Location Security
Vendors shall develop and document in detail the measures to be
taken in a central counting environment. These measures shall include
physical and procedural controls related to the:
a. Handling of ballot boxes,
b. Preparing of ballots for counting,
c. Counting operations, and
d. Reporting data.

6.4 Software Security

Voting systems shall meet specific security requirements for the
installation of software and for protection against malicious software.
6.4.1 Software and Firmware Installation
The system shall meet the following requirements for installation
of software, including hardware with embedded firmware:
a. If software is resident in the system as firmware, the vendor
shall require and state in the system documentation that every device
is to be retested to validate each ROM prior to the start of elections
operations,
b. To prevent alteration of executable code, no software shall be
permanently installed or resident in the system unless the system
documentation states that the jurisdiction must provide a secure
physical and procedural environment for the storage, handling,
preparation, and transportation of the system hardware,
c. The system bootstrap, monitor, and device-controller software
may be resident permanently as firmware, provided that this firmware
has been shown to be inaccessible to activation or control by any means
other than by the authorized initiation and execution of the vote-
counting program, and its associated exception handlers,
d. The election-specific programming may be installed and resident
as firmware, provided that such firmware is installed on a component
(such as computer chip) other than the component on which the operating
system resides; and
e. After initiation of election day testing, no source code or
compilers or assemblers shall be resident or accessible.
6.4.2 Protection Against Malicious Software
Voting systems shall deploy protection against the many forms of
threats to which they may be exposed such as file and macro viruses,
worms, Trojan horses, and logic bombs. Vendors shall develop and
document the procedures to be followed to ensure that such protection
is maintained in a current status.

6.5 Telecommunications and Data Transmission

There are four areas that must be addressed by telecommunications
and data transmission security capabilities:
Access control for telecommunications capabilities,
Data integrity,
Detection and prevention of data interception, and
Protection against external threats to which commercial
products used by a voting system may be susceptible.
6.5.1 Access Control
Voting systems that use telecommunications to communicate between
system components and locations are subject to the same security
requirements governing access to any other system hardware, software,
and data function.
6.5.2 Data Integrity
Voting systems that use electrical or optical transmission of data
shall ensure the receipt of valid vote records is verified at the
receiving station. This should include standard transmission error
detection and correction methods such as checksums or message digest
hashes. Verification of correct transmission shall occur at the voting
system application level and ensure that the correct data is recorded
on all relevant components consolidated within the polling place prior
to the voter completing casting of his or her ballot.
6.5.3 Data Interception Prevention
Voting systems that use telecommunications as defined in Section 5
to communicate between system components and locations before the poll
site is officially closed shall:
a. Implement an encryption standard currently documented and
validated for use by an agency of the U.S. Federal Government; and
b. Provide a means to detect the presence of an intrusive process,
such as an Intrusion Detection System.
6.5.4 Protection Against External Threats
Voting systems that use public telecommunications networks shall
implement protections against external threats to which commercial
products used in the system may be susceptible.
6.5.4.1 Identification of COTS Products
Voting systems that use public telecommunications networks shall
provide system documentation that clearly identifies all COTS hardware
and software products and communications services used in the
development and/or operation of the voting system, including:
a. Operating systems,
b. Communications routers,
c. Modem drivers, and
d. Dial-up networking software.
Such documentation shall identify the name, vendor, and version
used for each such component.
6.5.4.2 Use of Protective Software
Voting systems that use public telecommunications networks shall
use

[[Page 18987]]

protective software at the receiving-end of all communications paths
to:
a. Detect the presence of a threat in a transmission,
b. Remove the threat from infected files/data,
c. Prevent against storage of the threat anywhere on the receiving
device,
d. Provide the capability to confirm that no threats are stored in
system memory and in connected storage media, and
e. Provide data to the system audit log indicating the detection of
a threat and the processing performed.
Vendors shall use multiple forms of protective software as needed
to provide capabilities for the full range of products used by the
voting system.
6.5.4.3 Monitoring and Responding to External Threats
Voting systems that use public telecommunications networks may
become vulnerable, by virtue of their system components, to external
threats to the accuracy and integrity of vote recording, vote counting,
and vote consolidation and reporting processes. Therefore, vendors of
such systems shall document how they plan to monitor and respond to
known threats to which their voting systems are vulnerable. This
documentation shall provide a detailed description, including
scheduling information, of the procedures the vendor will use to:
a. Monitor threats, such as through the review of assessments,
advisories, and alerts for COTS components issued by the Computer
Emergency Response Team (CERT), for which a current listing can be
found at http://www.cert.org, the National Infrastructure Protection Center (NIPC), for which a current listing can be found at http://

http://www.nipc.gov/warnings/warnings.htm, and the Federal Computer Incident

Response Capability (FedCIRC), for which additional information can be
found at http://www.fedcirc.gov/,

b. Evaluate the threats and, if any, proposed responses,
c. Develop responsive updates to the system and/or corrective
procedures,
d. Submit the proposed response to the ITAs and appropriate states
for approval, identifying the exact changes and whether or not they are
temporary or permanent,
e. After implementation of the proposed response is approved by the
state, assist clients, either directly or through detailed written
procedures, how to update their systems and/or to implement the
corrective procedures no later than one month before an election, and
f. Address threats emerging too late to correct the system at least
one month before the election, including:
(1) Providing prompt, emergency notification to the ITAs and the
affected states and user jurisdictions,
(2) Assisting client jurisdictions directly, or advising them
through detailed written procedures, to disable the public
telecommunications mode of the system, and
(3) After the election, modifying the system to address the threat,
submitting the modified system to an ITA and appropriate state
certification authority for approval, and assisting client
jurisdictions directly, or advising them through detailed written
procedures, to update their systems and/or to implement the corrective
procedures after approval.
6.5.5 Shared Operating Environment
Ballot recording and vote counting can be performed in either a
dedicated or non-dedicated environment. If ballot recording and vote
counting operations are performed in an environment that is shared with
other data processing functions, both hardware and software features
shall be present to protect the integrity of vote counting and of vote
data. Systems that use a shared operating environment shall:
a. Use security procedures and logging records to control access to
system functions,
b. Partition or compartmentalize voting system functions from other
concurrent functions at least logically, and preferably physically as
well,
c. Controlled system access by means of passwords, and restriction
of account access to necessary functions only, and
d. Have capabilities in place to control the flow of information,
precluding data leakage through shared system resources.
6.5.6 Access to Incomplete Election Returns and Interactive Queries
If the voting system provides access to incomplete election returns
and interactive inquiries before the completion of the official count,
the system shall:
a. For equipment that operates in a central counting environment,
be designed to provide external access to incomplete election returns
only if that access for these purposes is authorized by the statutes
and regulations of the using agency. This requirement applies as well
to polling place equipment that contains a removable memory module, or
that may be removed in its entirety to a central place for the
consolidation of polling place returns.
b. Use voting system software and its security environment designed
such that data accessible to interactive queries resides in an external
file, or database, that is created and maintained by the elections
software under the restrictions applying to any other output report,
namely, that:
(1) The output file or database has no provision for write-access
back to the system.
(2) Persons whose only authorized access is to the file or database
are denied write-access, both to the file or database, and to the
system.

6.6 Security for Transmission of Official Data Over Public
Communications Networks

DRE systems that transmit data over public telecommunications
networks face security risks that are not present in other DRE systems.
This section describes standards applicable to DRE systems that use
public telecommunications networks.
6.6.1 General Security Requirements for Systems Transmitting Data Over
Public Networks
All systems that transmit data over public telecommunications
networks shall:
a. Preserve the secrecy of a voter's ballot choices, and prevent
anyone from violating ballot privacy,
b. Employ digital signature for all communications between the vote
server and other devices that communicate with the server over the
network, and
c. Require that at least two authorized election officials activate
any critical operation regarding the processing of ballots transmitted
over a public communications network takes place, i.e. the passwords or
cryptographic keys of at least two employees are required to perform
processing of votes.
6.6.2 Voting Process Security for Casting Individual Ballots over a
Public Telecommunications Network
Systems designed for transmission of telecommunications over public
networks shall meet security standards that address the security risks
attendant with the casting of ballots from poll sites controlled by
election officials using voting devices configured and installed by
election officials and/or their vendor or contractor, and using in-
person authentication of individual voters.
6.6.2.1 Documentation of Mandatory Security Activities
Vendors of systems that cast individual ballots over a public

[[Page 18988]]

telecommunications network shall provide detailed descriptions of:
a. All activities mandatory to ensuring effective system security
to be performed in setting up the system for operation, including
testing of security before an election; and
b. All activities that should be prohibited during system setup and
during the time frame for voting operations, including both the hours
when polls are open and when polls are closed.
6.6.2.2 Capabilities to Operate During Interruption of
Telecommunications Capabilities
These systems shall provide the following capabilities to provide
resistance to interruptions of telecommunications service that prevent
voting devices at the poll site from communicating with external
components via telecommunications:
a. Detect the occurrence of a telecommunications interruption at
the poll site and switch to an alternative mode of operation that is
not dependent on the connection between poll site voting devices and
external system components,
b. Provide an alternate mode of operation that includes the
functionality of a conventional DRE machine without losing any single
vote,
c. Create and preserve an audit trail of every vote cast during the
period of interrupted communication and system operation in
conventional DRE system mode,
d. Upon reestablishment of communications, transmit and process
votes accumulated while operating in conventional DRE system mode with
all security safeguards in effect, and
e. Ensure that all safeguards related to voter identification and
authentication are not affected by the procedures employed by the
system to counteract potential interruptions of telecommunications
capabilities.

Volume I, Section 7

Table of Contents

7 Quality Assurance

7.1 Scope
7.2 General Requirements
7.3 Components from Third Parties
7.4 Responsibility for Tests
7.5 Parts & Materials Special Tests and Examinations
7.6 Quality Conformance Inspections
7.7 Documentation

7 Quality Assurance

7.1 Scope

Quality Assurance provides continuous confirmation that a voting
system conforms with the Standards and to the requirements of state and
local jurisdictions. Quality Assurance is a vendor function with
associated practices that is initiated prior to system development and
continues throughout the maintenance life cycle of the voting system.
Quality Assurance focuses on building quality into a system and
reducing dependence on system tests at the end of the life cycle to
detect deficiencies, thus helping ensure the system:
Meets stated requirements and objectives;
Adheres to established standards and conventions;
Functions consistent with related components and meets
dependencies for use within the jurisdiction; and
Reflects all changes approved during its initial
development, internal testing, qualification, and, if applicable,
additional certification processes.

7.2 General Requirements

The voting system vendor is responsible for designing and
implementing a quality assurance program to ensure that the design,
workmanship, and performance requirements of this standard are achieved
in all delivered systems and components. At a minimum, this program
shall:
a. Include procedures for specifying, procuring, inspecting,
accepting, and controlling parts and raw materials of the requisite
quality;
b. Require the documentation of the hardware and software
development process;
c. Identify and enforce all requirements for:
(1) In-process inspection and testing that the manufacturer deems
necessary to ensure proper fabrication and assembly of hardware, and
(2) Installation and operation of software (including firmware).
d. Include plans and procedures for post-production environmental
screening and acceptance test; and
e. Include a procedure for maintaining all data and records
required to document and verify the quality inspections and tests.

7.3 Components from Third Parties

A vendors who does not manufacture all the components of its voting
system, but instead procures components as standard commercial items
for assembly and integration into a voting system, should verify that
the supplier vendors follow documented quality assurance procedures
that are at least as stringent as those used internally by the voting
system vendor.

7.4 Responsibility for Tests

The manufacturer or vendor shall be responsible for:
a. Performing all quality assurance tests;
b. Acquiring and documenting test data; and
c. Providing test reports for review by the ITA, and to the
purchaser upon request.

7.5 Parts & Materials Special Tests and Examinations

In order to ensure that voting system parts and materials function
properly, vendors shall:
a. Select parts and materials to be used in voting systems and
components according to their suitability for the intended application.
Suitability may be determined by similarity of this application to
existing standard practice, or by means of special tests;
b. Design special tests, if needed, to evaluate the part or
material under conditions accurately simulating the actual operating
environment; and
c. Maintain the resulting test data as part of the quality
assurance program documentation.

7.6 Quality Conformance Inspections

The vendor performs conformance inspections to ensure the overall
quality of the voting system and components delivered to the ITA for
testing and to the jurisdiction for implementation. To meet the
conformance inspection requirements the vendor or manufacturer shall:
a. Inspect and test each voting system or component to verify that
it meets all inspection and test requirements for the system; and
b. Deliver a record of tests, or a certificate of satisfactory
completion, with each system or component.

7.7 Documentation

Vendors are required to produce documentation to support the
development and formal testing of voting systems. To meet documentation
requirements, vendors shall provide complete product documentation with
each voting systems or components, as described Volume II, Section 2
for the TDP. This documentation shall:
a. Be sufficient to serve the needs of the ITA, voters, election
officials, and maintenance technicians;
b. Be prepared and published in accordance with standard industrial
practice for information technology and electronic and mechanical
equipment; and
c. Consist, at a minimum, of the following:
(1) System overview;

[[Page 18989]]

(2) System functionality description;
(3) System hardware specification;
(4) Software design and specifications;
(5) System security specification;
(6) System test and verification specification;
(7) System operations procedures;
(8) System maintenance procedures;
(9) Personnel deployment and training requirements;
(10) Configuration management plan;
(11) Quality assurance program; and
(12) System Change Notes.

Volume I, Section 8

Table of Contents

8 Configuration Management

8.1 Scope
8.1.1 Configuration Management Requirements
8.1.2 Organization of Configuration Management Standards
8.1.3 Application of Configuration Management Requirements
8.2 Configuration Management Policy
8.3 Configuration Identification
8.3.1 Structuring and Naming Configuration Items
8.3.2 Versioning Conventions
8.4 Baseline, Promotion, and Demotion Procedures
8.5 Configuration Control Procedures
8.6 Release Process
8.7 Configuration Audits
8.7.1 Physical Configuration Audit
8.7.2 Functional Configuration Audit
8.8 Configuration Management Resources

8 Configuration Management

8.1 Scope

This section contains specific requirements for configuration
management of voting systems. For the purpose of the Standards,
configuration management is defined as a set of activities and
associated practices that ensures full knowledge and control of the
components of a system, starting with its initial development and
progressing through its ongoing maintenance and enhancement. This
section describes activities in terms of their purposes and outcomes.
It does not describe specific procedures or steps to be employed to
accomplish them. Specific steps and procedures are left to the vendor
to select.
Vendors are required to submit these procedures to the Independent
Test Authority (ITA) as part of the Technical Data Package (TDP) for
system qualifications described in Volume II, Voting Systems
Qualification Testing Standards, for review against the requirements of
this section. Additionally, state or local election legislation,
regulations, or contractual agreements may require the vendor to
conform to additional standards for configuration management or to
adopt specific required procedures. Further, authorized election
officials or their representatives reserve the right to inspect vendor
facilities and operations to determine conformance with the vendor's
reported procedures and with any additional requirements.
8.1.1 Configuration Management Requirements
Configuration management addresses a broad set of recordkeeping,
audit, and reporting activities that contribute to full knowledge and
control of a system and its components. These activities include:
Identifying discrete system components;
Creating records of a formal baseline and later versions
of components;
Controlling changes made to the system and its components;
Releasing new versions of the system to ITAs;
Releasing new versions of the system to customers;
Auditing the system, including its documentation, against
configuration management records;
Controlling interfaces to other systems; and
Identifying tools used to build and maintain the system.
8.1.2 Organization of Configuration Management Standards
The standards for configuration management presented in this
section include:
Application of configuration management requirements;
Configuration management policy;
Configuration identification;
Baseline, promotion, and demotion procedures;
Configuration control procedures;
Release process;
Configuration audits; and
Configuration management resources.
8.1.3 Application of Configuration Management Requirements
Requirements for configuration management apply regardless of the
specific technologies employed to all voting systems subject to the
Standards. These system components include:
a. Software components;
b. Hardware components;
c. Communications components;
d. Documentation;
e. Identification and naming and conventions (including changes to
these conventions) for software programs and data files;
f. Development and testing artifacts such as test data and scripts;
and
g. File archiving and data repositories.

8.2 Configuration Management Policy

The vendor shall describe its policies for configuration management
in the TDP. This description shall address the following elements:
a. Scope and nature of configuration management program activities;
and
b. Breadth of application of the vendor's policies and practices to
the voting system (i.e., extent to which policies and practices apply
to the total system, and extent to which policies and practices of
suppliers apply to particular components, subsystems, or other defined
system elements.

8.3 Configuration Identification

Configuration identification is the process of identifying, naming,
and acquiring configuration items. Configuration identification
encompasses all system components.
8.3.1 Structuring and Naming Configuration Items
The vendor shall describe the procedures and conventions used to:
a. Classify configuration items into categories and subcategories;
b. Uniquely number or otherwise identify configuration items; and
c. Name configuration items;
8.3.2 Versioning Conventions
When a system component is used to identify higher-level system
elements, a vendor shall describe the conventions used to:
a. Identify the specific versions of individual configuration items
and sets of items that are used by the vendor to identify higher level
system elements such as subsystems;
b. Uniquely number or otherwise identify versions; and
c. Name versions.

8.4 Baseline, Promotion, and Demotion Procedures

The vendor shall establish formal procedures and conventions for
establishing and providing a complete description of the procedures and
related conventions used to:
a. Establish a particular instance of a component as the starting
baseline;
b. Promote subsequent instances of a component to baseline status
as development progresses through to completion of the initial
completed version released to the ITAs for qualification testing; and
c. Promote subsequent instances of a component to baseline status
as the component is maintained throughout its life cycle until system
retirement (i.e., the system is no longer sold or maintained by the
vendor).

[[Page 18990]]

8.5 Configuration Control Procedures

Configuration control is the process of approving and implementing
changes to a configuration item to prevent unauthorized additions,
changes, or deletions. The vendor shall establish such procedures and
related conventions, providing a complete description of those
procedures used to:
a. Develop and maintain internally developed items;
b. Acquire and maintain third-party items;
c. Resolve internally identified defects for items regardless of
their origin; and
d. Resolve externally identified and reported defects (i.e., by
customers and ITAs).

8.6 Release Process

The release process is the means by which the vendor installs,
transfers, or migrates the system to the ITAs and, eventually, to its
customers. The vendor shall establish such procedures and related
conventions, providing a complete description of those used to:
a. Perform a first release of the system to an ITA;
b. Perform a subsequent maintenance or upgrade release of the
system, or a particular components, to an ITA;
c. Perform the initial delivery and installation of the system to a
customer, including confirmation that the installed version of the
system matches exactly the qualified system version; and
d. Perform a subsequent maintenance or upgrade release of the
system, or a particular component, to a customer, including
confirmation that the installed version of the system matches exactly
the qualified system version.

8.7 Configuration Audits

The Standards require two types of configuration audits: Physical
Configuration Audits (PCA) and Functional Configuration Audits (FCA).
8.7.1 Physical Configuration Audit
The PCA is conducted by the ITA to compare the voting system
components submitted for qualification to the vendor's technical
documentation. For the PCA, a vendor shall provide:
a. Identification of all items that are to be a part of the
software release;
b. Specification of compiler (or choice of compilers) to be used to
generate executable programs;
c. Identification of all hardware that interfaces with the
software;
d. Configuration baseline data for all hardware that is unique to
the system;
e. Copies of all software documentation intended for distribution
to users, including program listings, specifications, operations
manual, voter manual, and maintenance manual;
f. User acceptance test procedures and acceptance criteria; and
g. Identification of any changes between the physical configuration
of the system submitted for the PCA and that submitted for the FCA,
with a certification that any differences do not degrade the functional
characteristics; and
h. Complete descriptions of its procedures and related conventions
used to support this audit by:
(1) Establishing a configuration baseline of the software and
hardware to be tested; and
(2) Confirming whether the system documentation matches the
corresponding system components.
8.7.2 Functional Configuration Audit
The FCA is conducted by the ITA to verify that the system performs
all the functions described in the system documentation. The vendor
shall:
a. Completely describe its procedures and related conventions used
to support this audit for all system components;
b. Provide the following information to support this audit:
(1) Copies of all procedures used for module or unit testing,
integration testing, and system testing;
(2) Copies of all test cases generated for each module and
integration test, and sample ballot formats or other test cases used
for system tests; and
(3) Records of all tests performed by the procedures listed above,
including error corrections and retests.
In addition to such audits performed by ITAs during the system
qualification process, elements of this audit may also be performed by
state election organizations during the system certification process,
and individual jurisdictions during system acceptance testing.

8.8 Configuration Management Resources

Often, configuration management activities are performed with the
aid of automated tools. Assuring that such tools are available
throughout the system life cycle, including if the vendor is acquired
by or merged with another organization, is critical to effective
configuration management. Vendors may choose the specific tools they
use to perform the record keeping, audit, and reporting activities of
the configuration management standards. The resources documentation
standard provided below focus on assuring that procedures are in place
to record information about the tools to help ensure that they, and the
data they contain, can be transferred effectively and promptly to a
third party should the need arise. Within this context, a vendor is
required to develop and provide a complete description of the
procedures and related practices for maintaining information about:
a. Specific tools used, current version, and operating environment;
b. Physical location of the tools, including designation of
computer directories and files; and
c. Procedures and training materials for using the tools.

Volume I, Section 9

Table of Contents

9 Overview of Qualification Tests

9.1 Scope
9.2 Documentation Submitted by Vendor
9.3 Voting Equipment Submitted by Vendor
9.4 Testing Scope
9.4.1 Test Categories
9.4.1.1 Focus of Functionality Tests
9.4.1.2 Focus of Hardware Tests
9.4.1.3 Focus of Software Evaluation
9.4.1.4 Focus of System-Level Integration Tests
9.4.1.5 Focus of Vendor Documentation Examination
9.4.2 Sequence of Tests and Audits
9.5 Test Applicability
9.5.1 General Applicability
9.5.1.1 Hardware
9.5.1.2 Software
9.5.2 Modifications to Qualified Systems
9.5.2.1 General Requirements for Modifications
9.5.2.2 Basis for Limited Testing Determinations
9.6 Qualification Test Process
9.6.1 Pre-test Activities
9.6.1.1 Initiation of Testing
9.6.1.2 Pre-test Preparation
9.6.2 Qualification Testing
9.6.2.1 Qualification Test Plan
9.6.2.2 Qualification Test Conditions
9.6.2.3 Qualification Test Fixtures
9.6.2.4 Witness of System Build and Installation
9.6.2.5 Qualification Test Data Requirements
9.6.2.6 Qualification Test Practices
9.6.3 Qualification Report Issuance and Post-test Activities
9.6.4 Resolution of Testing Issues

9 Overview of Qualification Tests

9.1 Scope

This section provides an overview of the testing process for
qualification testing of voting systems. Qualification testing is the
process by which a voting system is shown to comply with the
requirements of the Standards and the requirements of its own design
and performance specifications.
Qualification testing encompasses the examination of software;
tests of hardware under conditions simulating the intended storage,
operation, transportation, and maintenance

[[Page 18991]]

environments; the inspection and evaluation of system documentation;
and operational tests to validate system performance and function under
normal and abnormal conditions. The testing also evaluates the
completeness of the vendor's developmental test program, including the
sufficiency of vendor tests conducted to demonstrate compliance with
stated system design and performance specifications, and the vendor's
documented quality assurance and configuration management practices.
The tests address individual system components or elements, as well as
the integrated system as a whole. Since 1994, qualification tests for
voting systems have been performed by Independent Test Authorities
(ITAs) certified by the National Association of State Election
Directors (NASED). NASED has certified an ITA for either the full scope
of qualification testing or a distinct subset of the total scope of
testing. The test process described in this section may be conducted by
one or more ITAs, depending on the nature of tests to be conducted and
the expertise of the certified ITAs.
Qualification testing is distinct from all other forms of testing,
including developmental testing by the vendor, certification testing by
a state election organization, and system acceptance testing by a
purchasing jurisdiction:
Qualification testing follows the vendor's developmental
testing;
Qualification testing provides an assurance to state
election officials and local jurisdictions of the conformance of a
voting system to the Standards as input to state certification of a
voting system and acceptance testing by a purchasing jurisdiction; and
Qualification testing may precede state certification
testing, or may be conducted in parallel as established by the
certification program of individual states.
Generally a voting system remains qualified under the standards
against which it was tested, as long as all modifications made to the
system are evaluated and passed by a certified ITA. The qualification
test report remains valid for as long as the voting system remains
unchanged from the last tested configuration. However, if a new threat
to a particular voting system is discovered, it is the prerogative of
NASED to determine which qualified voting systems are vulnerable,
whether those systems need to be retested, and the specific tests to be
conducted. In addition, when new standards supersede the standards
under which the system was qualified, it is the prerogative of NASED to
determine when systems that were qualified under the earlier standards
will lose their qualification, unless they are tested to meet current
standards.
The remainder of this section describes the documentation and
equipment required to be submitted by the vendor, the scope of
qualification testing, the applicability to voting system components,
and the flow of the test process.

9.2 Documentation Submitted by Vendor

The vendor shall submit to the ITA documentation necessary for the
identification of the full system configuration submitted for
evaluation and for the development of an appropriate test plan by the
ITA for system qualification testing.
One element of the documentation is the Technical Data Package
(TDP). The TDP contains information that that defines the voting system
design, method of operation, and related resources. It provides a
system overview and documents the system's functionality, hardware,
software, security, test and verification specifications, operations
procedures, maintenance procedures, and personnel deployment and
training requirements. It also documents the vendor's configuration
management plan and quality assurance program. If the system was
previously qualified, the TDP also includes the system change notes.
This documentation is used by the ITA in constructing the
qualification testing plan and is particularly important in
constructing plans for the re-testing of systems that have been
qualified previously. Re-testing of systems submitted by vendors that
consistently adhere to particularly strong and well documented quality
assurance and configuration management practices will generally be more
efficient than for systems developed and maintained using less rigorous
or less well documented practices. Volume II provides a detailed
description of the documentation required for the vendor's quality
assurance and configuration management practices used for the system
submitted for qualification testing.

9.3 Voting Equipment Submitted by Vendor

Vendors may seek to market a complete voting system or an
interoperable component of a voting system. Nevertheless, vendors shall
submit for testing the specific system configuration that is to be
offered to jurisdictions or that comprises the component to be marketed
plus the other components with which the vendor recommends that
component be used. The system submitted for testing shall meet the
following requirements:
a. The hardware submitted for qualification testing shall be
equivalent, in form and function, to the actual production versions of
the hardware units or the COTS hardware specified for use in the TDP;
b. The software submitted for qualification testing shall be the
exact software that will be used in production units;
c. Engineering or developmental prototypes are not acceptable,
unless the vendor can show that the equipment to be tested is
equivalent to standard production units in both performance and
construction; and
d. Benchmark directory listings shall be submitted for all
software/firmware elements (and associated documentation) included in
the vendor's release as they would normally be installed upon setup and
installation.

9.4 Testing Scope

The qualification test process is intended to discover
vulnerabilities that, should they appear in actual election use, could
result in failure to complete election operations in a satisfactory
manner.
Five types of focuses guide the overall qualification testing
process:
Operational accuracy in the recording and processing of
voting data, as measured by target error rate, for which the maximum
acceptable error rate is no more than one in ten million ballot
positions, with a maximum acceptable error rate in the test process of
one in 500,000 ballot positions (while it would be desirable that there
be an error rate of zero, if this had to be proven by a test, the test
itself would take an infinity of time);
Operational failures or the number of unrecoverable
failures under conditions simulating the intended storage, operation,
transportation, and maintenance environments for voting systems, using
an actual time-based period of processing test ballots;
System performance and function under normal and abnormal
conditions; and
Completeness and accuracy of the system documentation and
configuration management records to enable purchasing jurisdictions to
effectively install, test, and operate the system.
Qualification testing complements and evaluates the vendor's
developmental testing, including any beta testing. The ITA evaluates
the completeness of the vendor's

[[Page 18992]]

developmental test program, including the sufficiency of vendor tests
conducted to demonstrate compliance with the Standards as well as the
system's performance specifications. The ITA undertakes sample testing
of the vendor's test modules and also designs independent system-level
tests to supplement and check those designed by the vendor. Although
some of the qualification tests are based on those prescribed in the
Military Standards, in most cases the test conditions are less
stringent, reflecting commercial, rather than military, practice. The
ITA may use automated software testing tools to assist in this process
if they are available for the software under examination.
The procedure for disposition of system deficiencies discovered
during qualification testing is described in Volume II of the
Standards. This procedure recognizes that some but not necessarily all
operational malfunctions (apart from software logic defects) may result
in rejection. Basically, any defect that results in or may result in
the loss or corruption of voting data, whether through failure of
system hardware, software, or communication, through procedural
deficiency, or through deficiencies in security and audit provisions,
shall be cause for rejection. Otherwise, malfunctions that result from
failure to comply fully with other requirements of this standard will
not in every case warrant rejection. Specific failure definition and
scoring criteria are also contained in Volume II.
9.4.1 Test Categories
The qualification test procedure is presented in several parts:
Functionality testing;
Hardware testing;
Software evaluation;
System-level integration tests, including audits; and
Examination of documented vendor practices for quality
assurance and for configuration management.
In practice, there may be concurrent indications of hardware and
software function, or failure to function, during certain examinations
and tests. Operating tests of hardware partially exercise the software
as well and therefore supplement software qualification. Security tests
exercise hardware, software and communications capabilities.
Documentation review conducted during software qualification
supplements the review undertaken for system-level testing.
The qualification test procedures are presented in these categories
because test authorities frequently focus separately on each. The
following subsections provide information that test authorities need to
conduct testing.
Not all systems being tested are required to complete all
categories of testing. For example, if a previously-qualified system
has had hardware modifications, the system may be subject only to non-
operating environmental stress testing of the modified component, and a
partial system-level test. If a system consisting of general purpose
COTS hardware or one that was previously qualified has had
modifications to its software, the system is subject only to software
qualification and system-level tests, not hardware testing. However, in
all cases the system documentation and configuration management records
will be examined to confirm that they completely and accurately reflect
the components and component versions that comprise the voting system.
9.4.1.1 Focus of Functionality Tests
Functionality testing is performed to confirm the functional
capabilities of a voting system submitted for qualification. The ITA
designs and performs procedures to test a voting system against the
requirements outlined in Section 2. In order to best compliment the
diversity of the voting systems industry, this part of the
qualification testing process is not rigidly defined. Although there
are basic functionality testing requirements, additions or variations
in testing are appropriate depending on the system's use of specific
technologies and configurations, the system capabilities, and the
outcomes of previous testing.
9.4.1.2 Focus of Hardware Tests
Hardware testing begins with non-operating tests that require the
use of an environmental test facility. These are followed by operating
tests that are performed partly in an environmental facility and partly
in a standard test laboratory or shop environment.
The non-operating tests are intended to evaluate the ability of the
system hardware to withstand exposure to the various environmental
conditions incidental to voting system storage, maintenance, and
transportation. The procedures are based on test methods contained in
Military Standards (MIL-STD) 810D, modified where appropriate, and
include such tests as: Bench handling, vibration, low and high
temperature, and humidity.
The operating tests involve running the system for an extended
period of time under varying temperatures and voltages. This period of
operation ensures with confidence that the hardware meets or exceeds
the minimum requirements for reliability, data reading, and processing
accuracy contained in Section 3. The procedure emphasizes equipment
operability and data accuracy; it is not an exhaustive evaluation of
all system functions. Moreover, the severity of the test conditions, in
most cases, has been reduced from that specified in the Military
Standards to reflect commercial and industrial, rather than military
and aerospace, practice.
9.4.1.3 Focus of Software Evaluation
The software qualification tests encompass a number of interrelated
examinations, involving assessment of application source code for its
compliance with the requirements spelled out in Volume I, Section 4.
Essentially, the ITA will look at programming completeness,
consistency, correctness, modifiability, structuredness and
traceability, along with its modularity and construction. The code
inspection will be followed by a series of functional tests to verify
the proper performance of all system functions controlled by the
software.
The ITA may inspect COTS generated software source code in the
preparation of test plans and to provide some minimal scanning or
sampling to check for embedded code or unauthorized changes. Otherwise,
the COTS source code is not subject to the full code review and
testing. For purposes of code analysis, the COTS units shall be treated
as unexpanded macros.
9.4.1.4 Focus of System-Level Integration Tests
The functionality, hardware, and software qualification tests
supplement a fuller evaluation performed by the system-level
integration tests. System-level tests focus on these aspects jointly,
throughout the full range of system operations. They include tests of
fully integrated system components, internal and external system
interfaces, usability and accessibility, and security. During this
process election management functions, ballot-counting logic, and
system capacity are exercised. The process also includes the Physical
Configuration Audit (PCA) and the Functional Configuration Audit (FCA).
The ITA tests the interface of all system modules and subsystems
with each other against the vendor's specifications. Some, but not all,
systems use telecommunications capabilities as defined in Section 5.
For those systems that do use such capabilities, components that are
located at the poll site or separate vote counting site are tested for
effective

[[Page 18993]]

interface, accurate vote transmission, failure detection, and failure
recovery. For voting systems that use telecommunications lines or
networks that are not under the control of the vendor (e.g., public
telephone networks), the ITA tests the interface of vendor-supplied
components with these external components for effective interface, vote
transmission, failure detection, and failure recovery.
The security tests focus on the ability of the system to detect,
prevent, log, and recover from a broad range of security risks as
identified in Section 6. The range of risks tested is determined by the
design of the system and potential exposure to risk. Regardless of
system design and risk profile, all systems are tested for effective
access control and physical data security. For systems that use public
telecommunications networks, to transmit election management data or
official election results (such as ballots or tabulated results),
security tests are conducted to ensure that the system provides the
necessary identity-proofing, confidentiality, and integrity of
transmitted data. The tests determine if the system is capable of
detecting, logging, preventing, and recovering from types of attacks
known at the time the system is submitted for qualification. The ITA
may meet these testing requirements by confirming the proper
implementation of proven commercial security software.
The interface between the voting system and its users, both voters
and election officials, is a key element of effective system operation
and confidence in the system. At this time, general standards for the
usability of voting systems by the average voter and election officials
have not been defined, but are to be addressed in the next update of
the Standards. However, standards for usability by individual voters
with disabilities have been defined in Section 2.7 based on Section 508
of the Rehabilitation Act Amendments of 1998. Voting systems are tested
to ensure that an accessible voting station is included in the system
configuration and that its design and operation conforms with these
standards.
The Physical Configuration Audit (PCA) compares the voting system
components submitted for qualification to the vendor's technical
documentation and confirms that the documentation submitted meets the
requirements of the Standards. As part of the PCA, the ITA also
witnesses the build of the executable system to ensure that the
qualified executable release is built from the tested components.
The Functional Configuration Audit (FCA) is an exhaustive
verification of every system function and combination of functions
cited in the vendors' documentation. Through use, the FCA verifies the
accuracy and completeness of the system's TDP. The various options of
software counting logic that are claimed in the vendor's documentation
shall be tested during the system-level FCA. Generic test ballots or
test entry data for DRE systems, representing particular sequences of
ballot-counting events, will test the counting logic during this audit.
9.4.1.5 Focus of Vendor Documentation Examination
The ITA reviews the documentation submitted by the vendor to
evaluate the extent to which it conforms to the requirements outlined
in Sections 7 and 8 for vendor configuration and quality assurance
practices. The ITA also evaluates the conformance of other
documentation and information provided by the vendor with the vendor's
documented practices for quality assurance and configuration
management.
The Standards do not require on-site examination of the vendor's
quality assurance and configuration management practices during the
system development process. However, the ITA conducts several
activities while at the vendor site to witness the system build that
enable assessment of the vendor's quality assurance and configuration
management practices and conformance with them. These include surveys,
interviews with individuals at all levels of the development team, and
examination of selected internal work products such as system change
requests and problem tracking logs.
9.4.2 Sequence of Tests and Audits
There is no required sequence for performing the system
qualification tests and audits. For a new system, not previously
qualified, a test using the generic test ballot decks might be
performed before undertaking any of the more lengthy and expensive
tests or documentation review. The ITA or vendor may, however, schedule
the PCA, FCA, or other tests in any convenient order, provided that the
prerequisite conditions for each test have been met before it is
initiated.

9.5 Test Applicability

Qualification tests are conducted for new systems seeking initial
qualification as well as for systems that are modified after
qualification.
9.5.1 General Applicability
Voting system hardware, software, communications and documentation
are examined and tested to determine suitability for elections use.
Examination and testing addresses the broad range of system
functionality and components, including system functionality for pre-
voting, voting, and post-voting functions described in Section 2. All
products custom designed for election use shall be tested in accordance
with the applicable procedures contained in this section. COTS
hardware, system software and communications components with proven
performance in commercial applications other than elections, however,
are exempted from certain portions of the test as long as such products
are not modified for use in a voting system. Compatibility of these
products all other components of the voting system shall be determined
through functional tests integrating these products with the remainder
of the system.
9.5.1.1 Hardware
Specifically, the hardware test requirements shall apply in full to
all equipment used in a voting system with the exception of the
following:
a. Commercially available models of general purpose information
technology equipment that have been designed to an ANSI or IEEE
standard, have a documented history of successful performance for
relevant requirements of the standards, and have demonstrated
compatibility with the voting system components with which they
interface;
b. Production models of special purpose information technology
equipment that have a documented history of successful performance
under conditions equivalent to election use for relevant requirements
of the standards and that have demonstrated compatibility with the
voting system components with which they interface; and
c. Any ancillary devices that do not perform ballot definition,
election database maintenance, ballot reading, ballot data processing,
or the production of an official output report; and that do not
interact with these system functions (e.g.; modems used to broadcast
results to the press, printers used to generate unofficial reports, or
CRTs used to monitor the vote counting process).
This equipment shall be subject to functional and operating tests
performed during software evaluation and system-level testing. However,
it need not undergo hardware non-operating tests. If the system is

[[Page 18994]]

composed entirely of off-the-shelf hardware, then the system also shall
not be subject to the 48-hour environmental chamber segment of the
hardware operating tests.
9.5.1.2 Software
Software qualification is applicable to the following:
a. Application programs that control and carry out ballot
processing, commencing with the definition of a ballot, and including
processing of the ballot image (either from physical ballots or
electronically activated images), and ending with the system's access
to memory for the generation of output reports;
b. Specialized compilers and specialized operating systems
associated with ballot processing; and
c. Standard compilers and operating systems that have been modified
for use in the vote counting process.
Specialized software for ballot preparation, election programming,
vote recording, vote tabulation, vote consolidation and reporting, and
audit trail production shall be subjected to code inspection.
Functional testing of all these programs during software evaluation and
system-level testing shall exercise any specially tailored software
off-line from the ballot counting process (e.g.; software for preparing
ballots and broadcasting results).
9.5.2 Modifications to Qualified Systems
Changes introduced after the system has completed qualification
under these Standards or earlier versions of the national Voting System
Standards will necessitate further review.
9.5.2.1 General Requirements for Modifications
The ITA will determine tests necessary for to qualify the modified
system based on a review of the nature and scope of changes, and other
submitted information including the system documentation, vendor test
documentation, configuration management records, and quality assurance
information. Based on this review, the ITA may:
a. Determine that a review of all change documentation against the
baseline materials is sufficient for recommendation for qualification;
or
b. Determine that all changes must be retested against the
previously qualified version (this will include review of changes to
source code, review of all updates to the TDP, and a performance of
system-level and functional tests); or
c. Determine that the scope of the changes is substantial and will
require a complete retest of the hardware, software, and/or
telecommunications.
9.5.2.2 Basis for Limited Testing Determinations
The ITA may determine that a modified system will be subject only
to limited qualification testing if the vendor demonstrates that the
change does not affect demonstrated compliance with these Standards
for:
a. Performance of voting system functions;
b. Voting system security and privacy;
c. Overall flow of system control; and
d. The manner in which ballots are defined and interpreted, or
voting data are processed.
Limited qualification testing is intended to facilitate the
correction of defects, the incorporation of improvements, the
enhancement of portability and flexibility, and the integration of
vote-counting software with other systems and election software.

9.6 Qualification Test Process

The qualification test process may be performed by one or more ITAs
that together perform the full scope of tests required by the
Standards. Where multiple ITAs are involved, testing shall be conducted
first for the voting system hardware, firmware, and related
documentation; then for the system software and communications; and
finally for the integrated system as a whole. Voting system hardware
and firmware testing may be performed by one ITA independently of the
other testing performed by other ITAs. Testing may be coordinated
across ITAs so that hardware/firmware tested by one ITA can be used in
the overall system tests performed by another ITA.
Whether one or more ITAs are used, the testing generally consists
of three phases:
[ssdiam] Pre-test Activities;
[ssdiam] Qualification Testing; and
[ssdiam] Qualification Report Issuance and Post-test Activities.
9.6.1 Pre-test Activities
Pre-test activities include the request for initiation of testing
and the pre-test preparation.
9.6.1.1 Initiation of Testing
Qualification testing shall be conducted at the request of the
vendor, consistent with the provision of the Standards. The vendor
shall:
a. Request the performance of qualification testing from among the
certified ITAs,
b. Enter into formal agreement with the ITAs for the performance of
testing, and
c. Prepare and submit materials required for testing consistent
with the requirements of the Standards.
Qualification testing shall be conducted for the initial version of
a voting system as well as for all subsequent changes to the system
prior to release for sale or for installation. As described in Section
9.5.2, the nature and scope of testing for system changes or new
versions shall be determined by the ITA based on the nature and scope
of the modifications to the system and on the quality of system
documentation and configuration management records submitted by the
vendor.
9.6.1.2 Pre-test Preparation
Pre-test preparation encompasses the following activities:
a. The vendor shall prepare and submit a complete TDP to the ITA.
The TDP should consist of the items listed in Section 9.2 and specified
in greater detail in Standards Volume II;
b. The ITA shall perform an initial review of the TDP for
completeness and clarity and request additional information as
required;
c. The vendor shall provide additional information, if requested by
the ITA;
d. The vendor and ITA shall enter into an agreement for the testing
to be performed by the ITA in exchange for payment by the vendor; and
e. The vendor shall deliver to the ITA all hardware and software
needed to perform testing.
9.6.2 Qualification Testing
Qualification testing encompasses the preparation of a test plan,
the establishment of the appropriate test conditions, the use of
appropriate test fixtures, the witness of the system build and
installation, the maintenance of qualification test data, and the
evaluation of the data resulting from tests and examinations.
9.6.2.1 Qualification Test Plan
The ITA shall prepare a Qualification Test Plan to define all tests
and procedures required to demonstrate compliance with Standards,
including:
a. Verifying or checking equipment operational status by means of
manufacturer operating procedures;
b. Establishing the test environment or the special environment
required to perform the test;
c. Initiating and completing operating modes or conditions
necessary to evaluate the specific performance characteristic under
test;
d. Measuring and recording the value or range of values for the
characteristic to be tested, demonstrating expected performance levels;

[[Page 18995]]

e. Verifying, as above, that the equipment is still in normal
condition and status after all required measurements have been
obtained;
f. Confirming that documentation submitted by the vendor
corresponds to the actual configuration and operation of the system;
and
g. Confirming that documented vendor practices for quality
assurance and configuration management comply with the Standards.
A recommended outline for the test plan and the details of required
testing are contained in Standards Volume II.
9.6.2.2 Qualification Test Conditions
The ITA may perform Qualification tests in any facility capable of
supporting the test environment. The following practices shall be
employed:
a. Preparations for testing, arrangement of equipment, verification
of equipment status, and the execution of procedures shall be witnessed
by at least one independent, qualified observer, who shall certify that
all test and data acquisition requirements have been satisfied;
b. When a test is to be performed at ``standard'' or ``ambient''
conditions, this requirement shall refer to a nominal laboratory or
office environment, with a temperature in the range of 68 to 75 degrees
Fahrenheit, and prevailing atmospheric pressure and relative humidity;
and
c. Otherwise, all tests shall be performed at the required
temperature and electrical supply voltage, regulated within the
following tolerances:
(1) Temperature +/-4 degrees F
(2) Electrical supply voltage +/-2 vac.
9.6.2.3 Qualification Test Fixtures
ITAs may use test fixtures or ancillary devices to facilitate
qualification testing. These fixtures and devices may include
arrangements for automating the operation of voting devices and the
acquisition of test data:
a. For systems that use a light source as a means of detecting
voter selections, the generation of a suitable optical signal by an
external device is acceptable. For systems that rely on the physical
activation of a switch, a mechanical fixture with suitable motion
generators is acceptable;
b. ITAs may use a simulation device, and appropriate software, to
speed up the process of testing and eliminate human error in casting
test ballots, provided that the simulation covers all voting data
detection and control paths that are used in casting an actual ballot.
In the event that only partial simulation is achieved, then an
independent method and test procedure shall be used to validate the
proper operation of those portions of the system not tested by the
simulator; and
c. If the vendor provides a means of simulating the casting of
ballots, the simulation device is subject to the same performance,
reliability, and quality requirements that apply to the voting device
itself.
9.6.2.4 Witness of System Build and Installation
Although most testing is conducted at facilities operated by the
ITA, a key element of voting system testing shall be conducted at the
vendor site. The ITA responsible for testing voting system software,
telecommunications, and integrated system operation (i.e., system wide
testing) shall witness the final system build, encompassing hardware,
software and communications, and the version of associated records and
documentation. The system elements witnessed, including their specific
versions, shall become the specific system version that is recommended
for qualification.
9.6.2.5 Qualification Test Data Requirements
The following qualification test data practices shall be employed:
a. A test log of the procedure shall be maintained. This log shall
identify the system and equipment by model and serial number;
b. Test environment conditions shall be noted; and
c. All operating steps, the identity and quantity of simulated
ballots, annotations of output reports, the elapsed time for each
procedure step, and observations of equipment performance and, in the
case of non-operating hardware tests, the condition of the equipment
shall be recorded.
9.6.2.6 Qualification Test Practices
The ITA shall conduct the examinations and tests defined in the
Test Plan such that all applicable tests identified in Standards Volume
II are executed to determine compliance with the requirements in
Sections 2-8 of the Standards. The ITA shall evaluate data resulting
from examinations and tests, employing the following practices:
a. If any malfunction or data error is detected that would be
classified as a relevant failure using the criteria in Volume II, its
occurrence, and the duration of operating time preceding it, shall be
recorded for inclusion in the analysis of data obtained from the test,
and the test shall be interrupted;
b. If a malfunction is due to a defect in software, then the test
shall be terminated and system returned to the vendor for correction;
c. If the malfunction is other than a software defect, and if
corrective action is taken to restore the equipment to a fully
operational condition within 8 hours, then the test may be resumed at
the point of suspension;
d. If the test is suspended for an extended period of time, the ITA
shall maintain a record of the procedures that have been satisfactorily
completed. When testing is resumed at a later date, repetition of the
successfully completed procedures may be waived, provided that no
design or manufacturing change has been made that would invalidate the
earlier test results;
e. Any and all failures that occurred as a result of a deficiency
shall be classified as purged, and test results shall be evaluated as
though the failure or failures had not occurred, if the:
(1) Vendor submits a design, manufacturing, or packaging change
notice to correct the deficiency, together with test data to verify the
adequacy of the change;
(2) Examiner of the equipment agrees that the proposed change will
correct the deficiency; and
(3) Vendor certifies that the change will be incorporated into all
existing and future production units; and
f. If corrective action cannot be successfully taken as defined
above, then the test shall be terminated, and the equipment shall be
rejected.
9.6.3 Qualification Report Issuance and Post-Test Activities
Qualification report issuance and post-test activities encompass
the activities described below:
a. The ITA may issue interim reports to the vendor, informing the
vendor of the testing status, findings to date, and other information.
Such reports do not constitute official test reports for voting system
qualification;
b. The ITA shall prepare a Qualification Test Report that confirms
the voting has passed the testing conducted by the ITA. The ITA shall
include in the Qualification Test Report the date testing was
completed, the specific system version addressed by the report, the
version numbers of all system elements separately identified with a
version number by the vendor, and the scope of tests conducted. A
recommended outline for the test report is contained in Volume II;
c. Where a system is tested by multiple ITAs, each ITA shall
prepare a Qualification Test Report;
d. The ITA shall deliver the Qualification Test Report to the
vendor and to NASED;

[[Page 18996]]

e. NASED shall issue a single Qualification Number for the system
to the vendor and to the ITAs. The issuance of a Qualification Number
indicates that the system has been tested by certified ITAs for
compliance with the national test standards and qualifies for the
certification process of states that have adopted the national
standards;
f. This number applies to the system as a whole only for the
configuration and versions of the system elements tested by the ITAs
and identified in the Qualification Test Reports. The Qualification
Number does not apply to individual system components or untested
configurations; and
g. The Qualification Number is intended for use by the states and
their jurisdictions to support state and jurisdiction processes
concerning voting systems. States and their jurisdictions shall request
ITA Qualification Test Reports based on the Qualification Number as
part of their voting system certification and procurement processes
systems that rely on the Standards.
9.6.4 Resolution of Testing Issues
The NASED Voting Systems Board (the Board) is responsible for
resolving questions about the application of the Standards in the
testing of voting systems. The Secretariat for the Board will relay its
decisions to the NASED certified ITAs and voting system vendors. The
Federal Election Commission will monitor these decisions in order to
determine which of them, if any, should be reflected in a subsequent
version of the standards.

Table of Contents

Volume I, Appendix A

A Glossary for Voting Systems

A.1 Glossary
A.2 Sources
A.3 List of Associations
A.4 List of Deprecated Terms

Glossary for Voting Systems

This glossary contains terms from the VSS-2002 as well as the
inclusion of additional terms needed to understand voting and related
areas such as security, human factors, and testing. Each term includes
a definition and its source as well as an association, where
Source is the source from which the definition originates.
A list of these sources is found in section A.2.
Association is the domain for which the term applies,
e.g., voting, testing, security. There may be multiple domains
identified for a term. There is no relevance given to the order in
which the domains are listed. A list of these associations is found in
section A.3.
At this time, a term may contain multiple definitions. The intent
is to eventually select one definition per term, unless multiple
definitions are necessary to convey the appropriate meanings of the
term.
Some of the terms in the VSS-2002 have been deprecated due to
changes in voting systems, voting process and/or mandates in HAVA. A
list of these deprecated terms is in section A.4 List of Deprecated
Terms.

A.1 Glossary

A
Abandoned Ballot: Ballot that the voter did not cast into the
ballot box or record vote on DRE before leaving the polling place. See
also fled voter.
Association: Voting.
Source: No attribution.

Absentee Ballot: Ballot prepared or designed for an absentee voter.
Definition of an absentee ballot is jurisdiction dependent.
Association: Voting.
Source: No attribution.

Acceptance Testing: Examination of a voting system and its
components by the purchasing election authority (usually in a
simulated-use environment) to validate performance of delivered units
in accordance with procurement requirements, and to validate that the
delivered system is, in fact, the certified or qualified system
purchased.
Association: Testing, voting.
Source: VSS.

Access Board: Independent federal agency devoted to accessibility
for people with disabilities.
Association: Human factors, HF: Accessibility.
Source: No attribution.

Accessibility: Measurable characteristic that indicates the degree
to which a system is available to, and usable by, individuals with
disabilities. The most common disabilities include those associated
with vision, hearing and mobility, as well as cognitive disabilities.
The HAVA also includes accessibility requirements for Native American
and Alaska Native citizens and alternative language access for voters
with limited English proficiency.
Association: Human factors, HF: Accessibility.
Source: NIST HF Rpt, HAVA.

Accessible Voting Station (Acc-VS): Voting Station equipped for
individuals with disabilities referred to in HAVA 301(a)(3)(B)
Association: HF: Accessibility, voting.
Source: HAVA.

Accreditation: (1) Formal recognition that a laboratory is
competent to carry out specific tests or calibrations or types of tests
or calibrations. (2) Procedure by which an authoritative body gives
formal recognition that a body or person is competent to carry out
specific tasks.
Association: Testing, standardization.
Source: (1) NIST HB 150, (2) ISO Guide 2-6.

Accreditation Body: (1) Authoritative body that performs
accreditation. (2) An independent organization responsible for
assessing the performance of other organizations against a recognized
standard, and for formally confirming the status of those that meet the
standard.
Association: Testing, conformity assessment.
Source: (1) ISO 17000, (2) IEEE 1583.

Accuracy: (1) Extent to which a given measurement agrees with an
accepted standard for that measurement. (2) Closeness of the agreement
between the result of a measurement and a true value of the particular
quantity subject to measurement.

Note 1: Accuracy is a qualitative concept. NOTE 2: The term
precision should not be used for accuracy.

Association: Testing.
Source: (1) IEEE 1583, (2) VIM.

Accuracy for Voting Systems: Ability of the system to capture,
record, store, consolidate and report the specific selections and
absence of selections, made by the voter for each ballot position
without error. Required accuracy is defined in terms of an error rate
that for testing purposes represents the maximum number of errors
allowed while processing a specified volume of data.
Association: Voting, testing.
Source: VSS.

Adequate Security: Security commensurate with the risk and the
magnitude of harm resulting from the loss, misuse, or unauthorized
access to or modification of information. See also risk assessment.
Association: Computer security.
Source: OMB A130.

Alternative Formats: In the context of voting systems, the ballot
or accompanying information is said to be in an alternative format if
it is in a representation other than the written English normally
displayed to non-disabled English-literate voters. NOTE: The usual
purpose of these formats is to provide accessibility to voters with
disabilities or those with limited English proficiency. Examples
include, but are not limited to, Braille, ASCII text, large print,
recorded audio, and electronic formats that comply with Part 1194 of
the standards for Section 508 of the Rehabilitation Act Amendments.

[[Page 18997]]

Association: HF: Accessibility.
Source: IEEE 1583, Section 508.

Alternative Language Voting Station (ALVS): voting station designed
to be usable by voters who have limited English proficiency, i.e.,
cannot read English.
Association: HF: Accessibility, voting.
Source: No attribution.

Approval: Permission for a product or process to be marketed or
used for stated purposes or under stated conditions. NOTE: Approval can
be based on fulfillment of specified requirements or completion of
specified procedures.
Association: Testing, conformity assessment.
Source: ISO 17000.

Attestation: Issue of a statement, based on a decision following
review, that fulfillment of specified requirements has been
demonstrated. NOTE: The resulting statement is also known as a
statement of conformity.
Association: Testing, conformity assessment.
Source: ISO 17000.

Audio Ballot: Voter interface which provides the voter with audio
stimuli and allows the voter to communicate intent to the voting system
through vocalization or physical actions. See also ballot.
Association: Voting, human factors, HF: Accessibility.
Source: FL Statutes.

Audio-Tactile Interface (ATI): Voter interface designed so as not
to require visual reading of a ballot. Audio is used to convey
information to the voter and sensitive tactile controls allow the voter
to convey information to the voting system.
Association: HF: Accessibility, voting.
Source: No attribution.

Audit: Systematic, independent, documented process for obtaining
records, statements of fact or other relevant information and assessing
them objectively to determine the extent to which specified
requirements are fulfilled. NOTE: While audit applies to management
systems, assessment applies to conformity assessment bodies as well as
more generally.
Association: Testing, conformity assessment, security.
Source: ISO 17000.

Audit Trail: Recorded information that allows election officials to
view the steps that occurred on the equipment included in an election
to verify or reconstruct the steps followed without compromising the
ballot or voter secrecy.
Association: Voting, security.
Source: No attribution.

Audit Trail for DRE: Paper printout of votes cast, produced by
direct response electronic (DRE) voting machines, which election
officials may use to crosscheck electronically tabulated totals.
Association: Voting, security.
Source: NASS.

Availability: Ensuring timely and reliable access to and use of
information.
Association: Security.
Source: 44 U.S.C.
B
Ballot: (1) Physical record of the selections made by a voter in
all of the races or contests in a particular election. Typically used
in the context of hand-counted paper, punched card, or optical mark-
sense ballots. When the ballot is recorded in electronic form, the term
ballot image is preferred. (2) An official presentation of all of the
contests to be decided in a particular election. These may be printed
on the ballot (sense 1), printed on a ballot label (as used for
punched-card and mechanical-lever voting machines), presented on a
computer display screen, or in some alternative form such as audio. See
also, audio ballot, ballot image, video ballot, electronic voter
interface.
Association: Voting.
Source: No attribution.

Ballot Configuration: Particular set of contests to appear on the
ballot for a particular election district, their order, the list of
ballot positions for each contest, and the binding of candidate names
to ballot positions.
Association: Voting.
Source: No attribution.

Ballot Counter: Counter in a voting device that counts the ballots
cast in a single election or election test.
Association: Voting.
Source: VSS.

Ballot Counting Logic: Software logic that defines the combinations
of voter choices that are valid and invalid on a given ballot and that
determines how the vote choices are totaled in a given election. States
differ from each other in the way they define valid and invalid votes
and in their vote-counting procedures.
Association: Voting.
Source: VSS.

Ballot Format: One of any number of specific ballot configurations
issued to the appropriate precinct. At a minimum, ballot formats differ
from one another in content. They may also differ in size of type,
graphical presentation, language used, or method of presentation (e.g.,
visual or audio). Also referred to as ballot style.
Association: Voting.
Source: VSS.

Ballot Image: (1) Electronically produced record of all votes cast
by a single voter. (2) Record of all votes produced by a single voter.
See also Cast Vote Record
Association: Voting.
Source: (1) VSS, (2) no attribution.

Ballot Instructions: The official instructional material presented
with the ballot (sense 2) to the voter. In some contexts, this is in
the form of an instructional poster in the voting booth, in some
contexts, as text on the ballot label, in any form, presented to voters
for expressing their selections in an election. This may be printed on
the ballot (sense 1), presented in audio form, posted in the voting
booth, printed on the ballot label or presented with the ballot
presentation.
Association: Voting.
Source: No attribution.

Ballot Measure: A contest on ballot where the voter may vote yes or
no. This term is typically used for referenda, amendments to state
constitutions and tax questions, but not for yes/no votes in judicial
retention races.
Association: Voting.
Source: No attribution.

Ballot Preparation: Process of using election databases or other
means to select the specific contests and questions to be contained in
a ballot format and related instructions; preparing and testing
election-specific software containing these selections; producing all
possible ballot formats; and validating the correctness of ballot
materials and software containing these selections for an upcoming
election.
Association: Voting.
Source: VSS.

Ballot Position: Abstract choice that is represented by a single
line item where a vote may be recorded in a ballot or ballot image.
Association: Voting.
Source: VSS.

Ballot Production: Process of converting the ballot format to a
medium ready for use in the physical ballot production or electronic
presentation.
Association: Voting.
Source: VSS.

Ballot Rotation: Process of varying the order of the candidate
names within a given contest to reduce the impact of voter bias towards
the candidate(s) listed first.
Association: Voting.
Source: VSS.

Ballot Set: See ballot image.
Association: Voting.

[[Page 18998]]

Source: VSS.

Ballot Scanner: Device used to read the data from a marksense
ballot.
Association: Voting.
Source: VSS.

Ballot Style: See ballot format.
Association: Voting.
Source: VSS.

Baseline: Product configuration that has been formally submitted
for review against the VVSG, which thereafter serves as the basis for
further development; and can be changed and offered to jurisdictions
only through formal change control and requalification procedures (and/
or recertification procedures where applicable).
Association: Voting, testing.
Source: VSS.
C
Calibration: Set of operations that establish, under specified
conditions, the relationship between values indicated by a measuring
instrument or measuring system, or values represented by a material
measure, and the corresponding known values of a quantity intended to
be measured.
Association: Testing.
Source: NIST HB 150.

Candidate: Person contending in a race for office. A candidate may
be explicitly presented as one of the choices on the ballot or may be a
write-in candidate.
Association: Voting.
Source: NIST HF Rpt.

Candidate Register: Record that reflects the total votes cast for
the candidate. This record is augmented as each ballot is cast on a DRE
or as digital signals from the conversion of voted paper ballots are
logically interpreted and recorded.
Association: Voting.
Source: VSS, IEEE 1583.

Canvass: (1) Compilation of election returns and validation of the
outcome that form the basis of the official results by political
subdivision. (2) Compilation of election returns for validation and
approval by the political subdivision of the outcome, which form the
basis for the official results.
Association: Voting.
Source: (1) VSS, IEEE 1583, (2) no attribution.

Cast Ballot: Ballot in which voter has taken final action in the
selection of candidates and measures and submits the ballot to the
appropriate jurisdiction.
Association: Voting.
Source: No attribution.

Cast Vote Record (CVR): Permanent record of all votes produced by a
single voter whether in electronic or paper copy form. Used for
counting votes. Also referred to as ballot set or ballot image when
used to refer to electronic ballots.
Association: Voting.
Source: (1) IEEE 1583.

Catastrophic System Failure: Total loss of function or functions,
such as the loss or unrecoverable corruption of voting data or the
failure of an on-board battery of volatile memory.
Association: Voting.
Source: VSS.

Central Counting: Counting of ballots in one or more locations
selected by the election authority for the processing or counting, or
both, of ballots.
Association: Voting.
Source: IL Statutes.

Certification: (1) Procedure by which a third party gives written
assurance that a product, process or service conforms to specified
requirements. (2) Third-party attestation related to products,
processes, systems or persons. See also State Certification and EAC
Certification.
Association: Testing, conformity assessment.
Source: (1) ISO Guide 2-6, (2) ISO 17000.

Certification Testing: Deprecated, replaced by State Certification.
Note: This term is being clarified with respect to testing State or
Federal Standards. See also EAC Certification.
Association: Testing, conformity assessment, voting.
Source: VSS.

Challenged Ballot: Ballot provided to individuals whose eligibility
to vote has been questioned. Once voted, such ballots are not included
in the tabulation until after the voter's eligibility is confirmed. See
also provisional ballot.
Association: Voting.
Source: VSS.

Checksum: Computed value representing the sum of the contents of an
instance of digital data; used to check whether errors have occurred in
transmission or storage.
Association: Security.
Source: No attribution.

Claim of Conformance: Statement by a vendor proclaiming that a
specific product conforms to a particular standard or set of standard
profiles, a claim which is verified or refuted by a testing authority.
Association: Testing, conformity assessment.
Source: No attribution.

Client: Any person or organization that engages the services of a
testing or calibration laboratory.
Association: Testing.
Source: NIST HB 150.

Closed Primary: Primary election in which voters receive a ballot
listing only those candidates running for office in the political party
with which the voters are affiliated, along with nonpartisan offices
and ballot issues presented at the same election.
Association: Voting.
Source: VSS.

Commercial Off-the-Shelf (COTS): Commercial, readily available
hardware devices (which may be electrical, electronic, mechanical,
etc.; such as card readers, printers, or personal computers) or
software products (such as operating systems, programming language
compilers, database management systems, subsystems, components;
software, etc.).
Association: IT.
Source: VSS, IEEE 1583.

Common Industry Format (CIF): Refers to the format described in
ANSI/INCITS 354-2001 ``Common Industry Format (CIF) for Usability Test
Reports.''
Association: HF: Usability.
Source: ANSI 354.

Compliance point: Identified, testable requirement.
Association: Testing, conformity assessment.
Source: No attribution.

Component: (1) Element within a larger system; a component can be
hardware or software. For hardware, a physical part of a subsystem that
can be used to compose larger systems (e.g., circuit boards, internal
modems, processors, computer memory). For software, a module of
executable code that performs a well-defined function and interacts
with other components. (2) Individual elements or items that
collectively comprise a device, e.g., circuit boards, internal modems,
processors, disk drives, and computer memory.
Association: IT.
Source: (1) No attribution, (2) VSS.

Confidentiality: (1) Prevention of unauthorized disclosure of
information. (2) Preserving authorized restrictions on information
access and disclosure, including means for protecting personal privacy
and proprietary information.
Association: Security.
Source: (1) IEEE 1583, (2) 44 U.S.C.

Configuration Identification: Element of configuration management,
consisting of selecting the configuration items for a system and
recording their functional and physical characteristics in technical
documentation.
Association: Testing, software engineering.

[[Page 18999]]

Source: IEEE 1583.

Configuration Item: Aggregation of hardware, software, or both that
is designated for configuration management and treated as a single
entity in the configuration management process.
Association: Testing, software engineering.
Source: IEEE 1583.

Configuration Management: Discipline applying technical and
administrative direction and surveillance to identify and document
functional and physical characteristics of a configuration item,
control changes to these characteristics, record and report change
processing and implementation status, and verify compliance with
specified requirements.
Association: Testing, software engineering.
Source: IEEE 1583.

Configuration Management Plan: Document detailing the process for
identifying, controlling and managing various released items (code,
hardware, documentation etc.)
Association: Testing, software engineering.
Source: IEEE 1583.

Conformance: See conformity
Association: Testing, standardization.
Source: No attribution.

Conformance Testing: Process of testing an implementation against
the requirements specified in one or more standards. The outcomes of a
conformance test are generally a pass or fail result, possibly
including reports of problems encountered during the execution. Also
known as conformity assessment.
Association: Testing, standardization.
Source: NIST HB 150.

Conformity: Fulfillment by a product, process or service of
specified requirements.
Association: Testing, standardization.
Source: ISO Guide 2-6.

Conformity Assessment: Demonstration that specified requirements
relating to a product, process, system, person or body are fulfilled.
See also testing, inspection, certification, accreditation, conformity
assessment bodies.
Association: Testing, standardization.
Source: ISO 17000.

Conformity Assessment Body: Body that performs conformity
assessment services. NOTE: An accreditation body is not a conformity
assessment body.
Association: Testing, standardization.
Source: ISO 17000.

Consensus: General agreement, characterized by the absence of
sustained opposition to substantial issues by any important part of the
concerned interests and by a process that involves seeking to take into
account the views of all parties concerned and to reconcile any
conflicting arguments.
Association: Standardization.
Source: ISO Guide 2-4.

Contest: Decision to be made within an election, which may be a
race for office or a referendum, propositions and/or questions. A
single ballot may contain one or more contests.
Association: Voting.
Source: No attribution.

Count: Process of totaling votes.
Association: Voting.
Source: VSS, IEEE 1583.

Counted Ballot: Ballot that has been processed and whose votes are
included in the candidate and measures vote totals.
Association: Voting.
Source: No attribution.

Corrective Action: Action taken to eliminate the causes of an
existing deficiency or other undesirable situation in order to prevent
recurrence.
Association: Testing.
Source: NIST HB 143.

Cross Filing: See Cross-party Endorsement.
Association: Voting.
Source: VSS.

Cross-party Endorsement: Endorsement of a single candidate or slate
of candidates by more than one political party. The candidate or slate
appears on the ballot representing each endorsing political party. Also
referred to as cross filing.
Association: Voting.
Source: VSS, IEEE 1583.

Cryptographic Key: Value used to control cryptographic operations,
such as decryption, encryption, signature generation or signature
verification.
Association: Security.
Source: NIST SP 800-63.

Cryptography: Discipline that embodies the principles, means, and
methods for the transformation of data in order to hide their semantic
content, prevent their unauthorized use, or prevent their undetected
modification.
Association: Security.
Source: NIST SP 800-59.

Cumulative Voting: Practice where voters are permitted to cast as
many votes as there are seats to be filled. Voters are not limited to
giving only one vote to a candidate. Instead, they can put multiple
votes on one or more candidates.
Association: Voting.
Source: VSS, IEEE 1583.
D
Data Accuracy: (1) Data accuracy is defined in terms of ballot
position error rate. This rate applies to the voting functions and
supporting equipment that capture, record, store, consolidate and
report the specific selections, and absence of selections, made by the
voter for each ballot position. (2) The system's ability to process
voting data absent internal errors generated by the system. It is
distinguished from data integrity, which encompasses errors introduced
by an outside source.
Association: Testing, security.
Source: (1) VSS, (2) IEEE 1583.

Data Integrity: Invulnerability of the system to accidental
intervention or deliberate, fraudulent manipulation that would result
in errors in the processing of data. It is distinguished from data
accuracy that encompasses internal, system-generated errors.
Association: Security.
Source: IEEE 1583.

Decertification: Withdrawal of certification of voting system
hardware and software.
Association: Testing, conformity assessment.
Source: HAVA.

Design Entity: Component of a design, named and referenced
uniquely, that is both structurally and functionally different from
other elements.
Association: Software engineering.
Source: IEEE 1583.

Design Entity Attributes: Named characteristic or property of a
design entity, which provides a statement of fact about the entity.
Attributes define the design entity and not the design process.
Association: Software engineering.
Source: IEEE 1583.

Designating Authority: Body established within government or
empowered by government to designate conformity assessment bodies,
suspend or withdraw their designation or remove their suspension from
designation.
Association: Testing, conformity assessment.
Source: ISO 17000.

Designation: Governmental authorization of a conformity assessment
body to perform specified conformity assessment activities.
Association: Testing, conformity assessment.
Source: ISO 17000.

Device: Functional unit that performs its assigned tasks as an
integrated whole.
Association: IT.
Source: VSS.

Digital Signature: Asymmetric key operation where the private key
is used

[[Page 19000]]

to digitally sign an electronic document and the public key is used to
verify the signature. Digital signatures provide authentication and
integrity protection.
Association: Security.
Source: SP 800-63.

Direct Record Electronic (DRE) Voting System: Voting system that
records votes by means of a ballot display provided with mechanical or
electro-optical components that can be actuated by the voter, that
processes the data by means of a computer program, and that records
voting data and cast vote records in internal and/or external memory
components. It produces a tabulation of the voting data stored in a
removable memory component and/or in printed copy.
Association: Voting
Source: VSS, IEEE 1583.

Directly Verified: Voting system that allows the voter to verify at
least one representation of his or her ballot with his/her own senses,
not using any software or hardware intermediary. Examples of a directly
verified voting system include DRE with a voter verified paper trail or
marksense system. This is in contrast with an indirectly verified
voting system.
Association: Voting, security.
Source: No attribution.

Disability: Disability means, with respect to an individual, (a) a
physical or mental impairment that substantially limits one or more of
the major life activities of such individual, (b) a record of such an
impairment, or (c) being regarded as having such an impairment.
Association: Human factors, HF: Accessibility
Source: ADA.

DRE Display: Part of the DRE that displays the electronic record.
Association: Security, voting.
Source: No attribution.

DRE-VVPAT: DRE voting system containing VVPAT capability. See also
Direct Record Electronic Voting System and Voter Verified Paper Audit
Trail.
Association: Security, voting.
Source: No attribution.

Dynamic Voting System Software: Software that changes over time
once it is installed on the voting equipment. See also voting system
software.
Association: Voting.
Source: No attribution.
E
EAC: Election Assistance Commission.

Early Voting: Voter completes the ballot in person at a county
office or other designated polling site or ballot drop site prior to
Election Day. The ballot is cast and not retrievable. NOTE: Early
voting is not the same as absentee voting. Also known as Early In-
Person Voting.
Association: Voting.
Source: Electionline.

Election Coding: See Election Programming.
Association: Voting.
Source: IEEE 1583.

Election Databases: Data file or set of files that contain
geographic information about political subdivisions and boundaries, all
contests and questions to be included in an election, and the
candidates for each contest.
Association: Voting.
Source: VSS, IEEE 1583.

Election Definition: Abstract definition of the races and questions
that may appear on ballot forms.
Association: Voting.
Source: No attribution.

Election District: Geographic area represented by a public official
who is elected by voters residing within the district boundaries. The
district may cover an entire state or political subdivision, may be a
portion of the state or political subdivision, or may include portions
of more than one political subdivision.
Association: Voting.
Source: VSS, IEEE 1583.

Election Management System: Set of processing functions and
databases within a Voting System that define, develop and maintain
election databases, perform election definition and setup functions,
format ballots, count votes, consolidate and report results, and
maintain audit trails.
Association: Voting.
Source: VSS, IEEE 1583.

Election Officials: Term used to designate the group of people
associated with conducting an election, including election personnel
and poll workers.
Association: Voting.
Source: No attribution.

Election Programming: Process by which election officials or their
designees use voting system software to logically define the ballot for
a specific election.
Association: Voting.
Source: VSS, IEEE 1583.

Electronic Ballot Printer (EBP): DRE-like device that fully prints
paper-based ballots with selected vote choices for tabulation by a
separate ballot scanner.
Association: Voting.
Source: IEEE 1583.

Electronic Cast Vote Record (ECVR): Deprecated, replaced by Cast
Vote Record (CVR).
Association: Voting.
Source: IEEE 1583.

Electronic Vote Capture System (EVCS): Election system that
encompasses DREs as well as accessible ballot printers (ABPs) when they
are combined with the ballot scanner that processes the printed ballot.
See also Voter Verified Paper Audit.
Association: Voting.
Source: IEEE 1583.

Electronic Voter Interface: Subsystem within a DRE voting system
which communicates ballot information to a voter in video, audio or
Braille form and which allows the voter to select candidates and issues
by means of vocalization or physical actions.
Association: Voting, Human factors, HF: Accessibility.
Source: FL Statutes.

Electronic Voting Machine: Any system that utilizes an electronic
component. Term is generally used to refer to DREs. See also Voting
Equipment, Voting System.
Association: Voting.
Source: NASS.

Electronically-Assisted Ballot Marker (EBM): Machines that provide
assistance to voters who are visually impaired, who have difficulty
reading English, or in other cases where a voter has difficulty
correctly marking by hand a preprinted paper ballot that is to be
counted in optical scan systems. The device marks, or helps to mark
selected vote choices on a previously inserted, preprinted paper
ballot. The machine then provides audio, tactile, or visual feedback to
the voter on what choices they have made on the ballot. The resulting
ballots are later tabulated on the same unit that processes ordinary
hand-marked paper ballots.
Association: Voting, human factors.
Source: IEEE 1583.

Entity Relationship Diagram (ERD): A data modeling technique that
creates a graphical representation of the entities, and the
relationships between entities, within an information system.
Association: Software engineering.
Source: IEEE 1583.

Error correction code: Coding system that incorporates extra parity
bits in order to detect errors.
Association: Security.
Source: WordNet.

E-Voting: (1) Term frequently used to refer to DREs and other types
of electronic voting equipment, but may be misleading as it implies
remote access via a computer network or the Internet. (2) Election
system that allows a voter to record his or her secure and secret
ballot electronically. See also DRE, Electronic Voting Machine.

[[Page 19001]]

Association: Voting.
Source: (1) NASS, (2) Whatis.com.
F
Federal Information Processing Standard (FIPS): Standard for
adoption and use by federal agencies that has been developed within the
National Institute of Standards and Technology (NIST) Information
Technology Laboratory and published by NIST, an part of the U.S.
Department of Commerce.
Association: Security, standardization.
Source: No attribution.

Firmware: Computer programs (software) stored in read-only memory
(ROM) devices embedded in the system and not capable of being altered
during system operation.
Association: IT.
Source: IEEE 1583.

Fled Voter: Voter who has begun the process of using voting
equipment to cast a ballot and has exited the polling site without
completing the casting of the ballot, thereby leaving the voting
equipment in a state in which election procedures must be used to
decide whether the fled voter's incomplete ballot will be cast before
the voting equipment is reset. See also abandoned ballot.
Association: Voting.
Source: No attribution.

Font: Family or assortment of characters of a given size and style,
e.g., 9-point Bodoni modern. See type font.
Association: Human factors, typography.
Source: ANSI Dict.

Functional Configuration Audit (FCA): Exhaustive verification of
every system function and combination of functions cited in the
vendor's documentation. Through use the FCA verifies the accuracy and
completeness of the system's Voter Manual, Operations Procedures,
Maintenance Procedures, and Diagnostic Testing Procedures.
Association: testing, voting.
Source: VSS, IEEE 1583.

Functional Test: Test performed to verify or validate the
accomplishment of a function or a series of functions.
Association: Testing.
Source: VSS, IEEE 1583.
G
General Election: Election in which voters, regardless of party
affiliation, are permitted to select persons to fill public office and
vote on ballot issues. Where the public office may be filled by a
candidate affiliated with a political party or when permitted by law,
unaffiliated candidate and voters choose among the candidates.
Association: Voting.
Source: VSS, IEEE 1583.
H
Hash: Algorithm that maps a bit string of arbitrary length to a
fixed-length bit string. Approved hash functions satisfy the following
properties: (a) It is computationally infeasible to find any input that
map to any prespecified output, and (b) it is computationally
infeasible to find any two distinct inputs that map to the same output.
Association: Voting.
Source: NIST SP 800-63.

HAVA: Help America Vote Act of 2002.
Association: Voting.
Source: No attribution.

Human Computer Interaction: Discipline concerned with the design,
evaluation and implementation of interactive computing systems for
human use and with the study of major phenomena surrounding them.
Association: Human factors.
Source: ACM SIGCHI.

Human Factors (or Ergonomics): Scientific discipline concerned with
the understanding of interactions among humans and other elements of a
system, and the profession that applies theory, principles, data and
methods to design in order to optimize human well-being and overall
system performance.
Association: Human factors.
Source: IEA.
I
Indirectly Verified: Voting system that allows a voter to verify
the ballot produced by his or her vote only via hardware or software
intermediary. An example of an indirectly verified voting system is a
touch screen DRE where the voter verifies the ballot through the
assistance of audio stimuli. This is in contrast to directly verified
voting systems.
Association: Voting, security.
Source: No attribution.

Implementation Conformance Statement: See Implementation Statement.
Implementation Statement: Statement by a vendor indicating the
capabilities, features, and optional functions as well as extensions
that have been implemented. Also known as implementation conformance
statement.
Association: Testing.
Source: No attribution.

Independent Testing Authority (ITA): Deprecated, replaced by Voting
System Testing Laboratory. Organization certified by the National
Association of State Election Directors (NASED) to perform
qualification testing.
Association: Testing, Voting.
Source: VSS.

Information Security: Protecting information and information
systems from unauthorized access, use, disclosure, disruption,
modification, or destruction in order to provide integrity,
confidentiality, and availability.
Association: Security.
Source: 44 U.S.C.

Inspection: Examination of a product design, product, process or
installation and determination of its conformity with specific
requirements or, on the basis of professional judgment, with general
requirements. NOTE: Inspection of a process may include inspection of
persons, facilities, technology and methodology.
Association: Testing, conformity assessment.
Source: ISO 17000.

Integrity: (1) Prevention of unauthorized modification of
information. (2) Guarding against improper information modification or
destruction, and includes ensuring information non-repudiation and
authenticity.
Association: Security.
Source: (1) IEEE 1583, (2) 44 U.S.C.
K
Key Management: Activities involving the handling of cryptographic
keys and other related security parameters (e.g., passwords) during the
entire life cycle of the keys, including their generation, storage,
establishment, entry and output, and zeroization.
Association: Security.
Source: FIPS 140-2.
L
Logic and Accuracy Testing: Testing of the tabulator setups of a
new election definition to ensure that the content correctly reflects
the election being held (i.e., contests, candidates, number to be
elected, ballot styles, etc.) and that all voting positions can be
voted for the maximum number of eligible candidates and that results
are accurately tabulated and reported.
Association: Voting, testing.
Source: IEEE 1583.

Logical Correctness: Condition signifying that, for a given input,
a computer program will satisfy the program specification (produce the
required output).
Association: Testing.
Source: VSS, IEEE 1583.
M
Marksense: System by which votes are recorded by means of marks
made in

[[Page 19002]]

voting response fields designated on one or both faces of a ballot card
or series of cards. Marksense systems may use an optical scanner or
similar sensor to read the ballots. Also known as Optical Scan.
Association: Voting.
Source: VSS, IEEE 1583.

Measure Register: Record that reflects the total votes cast for and
against a specific ballot issue. This record is augmented as each
ballot is cast on a DRE or as digital signals from the conversion of
voted paper ballots are logically interpreted and recorded.
Association: Voting.
Source: VSS, IEEE 1583.

Mechanical Lever Voting Machine: Machine that directly records a
voter's choices via mechanical level-actuated controls into a counting
mechanism that tallies the votes without using a physical ballot.
Association: Voting.
Source: ME Statutes.

Multi-seat Content: Contest in which multiple candidates can run,
up to a specified number of seats. Voters may vote for no more than the
specified number of candidates. Also known as field race.
Association: Voting.
Source: NIST HF Rpt.
N
NVLAP: The NIST National Voluntary Laboratory Accreditation
Program.
Association: Testing.
Source: No attribution.

Non-partisan Office: Elected office for which candidates run
independent of political party affiliation.
Association: Voting.
Source: VS, IEEE 1583.

Nonvolatile Memory: Memory in which information can be stored
indefinitely with no power applied. Static RAM, ROMs and EPROMs are
examples of nonvolatile memory.
Association: IT.
Source: VSS, IEEE 1583.
O
On-Site Absentee Voting: See Early Voting.

Open Primary: Primary election in which voters, regardless of
political affiliation, may choose in which party's primary they will
vote. Some states require voters to publicly declare their choice of
party ballot at the polling place, after which the poll worker provides
or activates the appropriate ballot. Other states allow the voters to
make their choice of party ballot within the privacy of the voting
booth. Voters also may be permitted to vote on nonpartisan offices and
ballot issues that are presented at the same election.
Association: Voting.
Source: VSS, IEEE 1583.

Operational Environment: See Voting Equipment Operational
Environment.
Association: Voting, IT.
Source: IEEE 1583.

Operations Procedures: See Voting Equipment Operations Procedures.
Association: Voting, IT.
Source: IEEE 1583.

Optical Scan, Optical Scan System: See Marksense.
Association: Voting.
Source: IEEE 1583.

Overvotes: (1) Generally prohibited practice of voting for more
than the allotted number of candidates for the office being contested.
(2) The voting for more than the allotted number of selections in a
race. (3) Occurs when the number of alternatives selected by a voter in
a contest exceeds the maximum number allowed for that contest. Also
known as overvoting.
Association: Voting.
Source: (1) VSS, (2) IEEE 1583, (3) NIST HF Rpt.
P
Paper-based Voting System: Voting system that records votes, counts
votes, and produces a tabulation of the vote count, using one or more
ballot cards or a written list of choices.
Association: Voting.
Source: VSS, IEEE 1583.

Paper Record: Paper ballot image or summary that is a copy of the
electronic record and that is verifiable by a voter. See also ballot
image.
Association: Voting, security.
Source: No attribution.

Partisan Office: Elected office for which (partisan and non-
partisan) candidates run as representatives of a political party.
Association: Voting.
Source: VSS, IEEE 1583.

Pass/Fail Criteria: Decision factor or expected result used to
determine if software or hardware passes a test case.
Association: Testing.
Source: IEEE 1583.

Physical Configuration Audit (PCA): (1) Inspection that compares
the voting system components submitted for qualification to the
vendor's technical documentation and confirms that the documentation
submitted meets the requirements of the VVSG. As part of the PCA, the
building of the executable system to ensure that the qualified
executable release is built from the tested components is also
witnessed. (2) Review, by the test authority, of the vendor's technical
documentation, source code, and observation of the code compile.
Association: Testing, voting.
Source: (1) VSS, (2) IEEE 1583.

Precinct Count: Counting of ballots on automatic tabulating
equipment provided by the election authority in the same precinct
polling place in which those ballots have been cast.
Association: Voting.
Source: IL Statutes.

Point Size: Method of measuring type, where the size of a font is
measured from the top of the tallest character to the bottom of the
lowest character.
Association: Human factors, typography.
Source: No attribution.

Political Subdivision: Any unit of government, such as counties and
cities but often excepting school districts, having authority to hold
elections for public offices or on ballot issues.
Association: Voting.
Source: VSS.

Polling Location: Physical address of a polling place.
Association: Voting.
Source: VSS, IEEE 1583.

Polling Place: Facility that is staffed by poll workers and
equipped with voting equipment, to which voters from a given precinct
come to cast in-person ballots. See also voting station.
Association: Voting.
Source: VSS, IEEE 1583.

Precinct: Administrative division representing a geographic area in
which voters cast ballots at the same polling place. Voters casting
absentee ballots may also be combined into one or more administrative
absentee precincts for purposes of tabulating and reporting votes.
Generally, voters in a polling place precinct are eligible to vote in a
general election using the same ballot format. In some jurisdictions,
however, the ballot formats may be different due to split precincts or
required ballot rotations within the precinct.
Association: Voting.
Source: VSS, IEEE 1583.

Precision: (1) Extent to which a given set of measurements of the
same sample agree with their mean. Thus, precision is commonly taken to
be the standard deviation estimated from sets of duplicate measurements
made under conditions of repeatability, that is, independent test
results obtained with the same method on identical test material, in
the same laboratory or test facility, by the same operator using the
same equipment in short intervals of time. (2) Degree of refinement in
measurement or specification, especially as represented by the number
of digits given.
Association: Testing, statistics.

[[Page 19003]]

Source: IEEE 1583.

Pre-Standard: Document that is adopted provisionally by a
standardizing body and made available to the public in order that the
necessary experience may be gained from its application on which to
base a standard.
Association: Standardization.
Source: ISO Guide 2-4.

Primary Election: Election held to determine which candidate will
represent a political party in the general election. Some states have
an open primary, while others have a closed primary. Sometimes
elections for nonpartisan offices and ballot issues are held during
primary elections.
Association: Voting.
Source: VSS.

Primary Presidential Delegation Nominations: Primary election in
which voters choose the delegates to the Presidential nominating
conventions allotted to their states by the national party committees.
Association: Voting.
Source: VSS.

Privacy: Voting system is said to provide privacy when it makes it
impossible for others to find out how the voter voted.
Association: Security, voting.
Source: No attribution.

Private Key: The secret part of an asymmetric key pair that is
typically used to digitally sign or decrypt data.
Association: Security.
Source: NIST SP 800-63.

Profile: (1) Subset of a standard for a particular constituency
that identifies the features, options, parameters, and implementation
requirements necessary for meeting a particular set of requirements.
(2) Specialization of a standard for a particular context, with
constraints and extensions that are specific to that context.
Association: Standardization.
Source: (1) ISO 8632, (2) no attribution.

Provisional Ballot: Ballot provided to individuals who claim they
are eligible to vote but whose eligibility cannot be confirmed when
they present themselves to vote. Once voted, such ballots are not
included in the tabulation until after the voter's eligibility is
confirmed. See also challenged ballot.
Association: Voting.
Source: VSS, IEEE 1583, NASS.

Public Information Package (PIP): Data to be published openly and
made available to all without let or hindrance, irrespective of need-
to-know.
Association: Testing.
Source: No attribution.

Public Key: Public part of an asymmetric key pair that is typically
used to verify signatures or encrypt data.
Association: Security.
Source: NIST SP 800-63.

Public Key Certificate: Digital document issued and digitally
signed by the private key of a Certification Authority that binds the
name of a subscriber to a public key. The certificate indicates that
the subscriber identified in the certificate has sole control and
access to the private key.
Association: Security.
Source: NIST SP 800-63.

Public Network Direct Record Electronic (DRE) Voting System: Form
of DRE voting system that uses electronic ballots and transmits vote
data from the polling place to another location (such as a central
count facility) over a public network beyond the control of the
election authority.
Association: Voting.
Source: VSS.

Punchcard Voting System: Voting system where votes are recorded by
means of punches made in voting response fields designated on one or
both faces of a ballot card or series of cards.
Association: Voting.
Source: VSS, IEEE 1583.
Q
Qualification Number: Deprecated. A number issued by NASED
(National Association of State Election Directors) to a system that has
been tested by certified Independent Test Authorities for compliance
with the qualification test standards. Issuance of a Qualification
Number indicates that the system qualifies for certification process of
states that have adopted the Standards. Note: Qualification Numbers for
Voting Systems that were qualified for compliance to the 1990 Voting
System Standards are still valid. Voting Systems that were qualified
for compliance to the Voting System Standards 2002 will need to be
assigned an EAC Certification number.
Association: Testing, voting.
Source: VSS.

Qualification Test Report: Deprecated, replaced by Test Report for
EAC Certification.
Association: Testing, voting.
Source: VSS, NIST HB150.

Qualification Testing: Examination and testing of a computerized
voting system by using qualification test standards to determine if the
system complies with the qualification performance and test standards
and with its own specifications. This process occurs prior to state
certification.
Association: Testing, voting.
Source: VSS.

Quality Assurance Plan: Document that identifies the system and
actions required to provide adequate assurance that an item or product
conforms to the documented technical requirements.
Association: Testing.
Source: IEEE 1583.

Quality Control: Operational techniques and activities that are
used to fulfill requirements for quality.
Association: Testing.
Source: NIST HB 150.

Quality Manual: Document stating the quality policy and describing
the quality system of an organization.
Association: Testing, software engineering.
Source: NIST HB 150.
R
Race: Contest between candidates.
Association: Voting.
Source: No attribution.

Ranked Order Voting: Practice that allows voters to rank candidates
in a contest in order of choice: 1, 2, 3 and so on. It takes a majority
to win. If anyone receives a majority of the first choice votes, that
candidate wins that election. If not, the last place candidate is
deleted, and all ballots are counted again, but this time each ballot
cast for the deleted candidate counts for the next choice candidate
listed on the ballot. The process of eliminating the last place
candidate and recounting the ballots continues until one candidate
receives a majority of the vote. The practice is also known as instant
runoff voting, preferences or preferential voting, or choice voting.
Association: Voting.
Source: VSS, IEEE 1583.

Read Ballot: Ballot that has been processed but may or may not be
counted.
Association: Voting.
Source: No attribution.

Recall Issue with Options: Process that allows voters to remove
their elected representatives from office prior to the expiration of
their terms of office. Often, the recall involves not only the question
of whether a particular officer should be removed from office, but also
the question of naming a successor in the event that there is an
affirmative vote for the recall.
Association: Voting.
Source: VSS.

Recertification: State examination, and possibly the retesting of a
voting system that was modified subsequent to

[[Page 19004]]

receiving state certification. The object of this process is to
determine if the modification still permits the system to function
properly in accordance with state requirements.
Association: Voting.
Source: VSS, IEEE.

Record: (n) Data that are preserved by a voting system, not
necessarily in any particular form. (v) To preserve such data.
Association: Voting.
Source: No attribution.

Records: Recordings of evidence of activities performed or results
achieved (e.g., forms, reports, test results), which serve as a basis
for verifying that the organization and the information system are
performing as intended. Also used to refer to units of related data
fields (i.e., groups of data fields that can be accessed by a program
and that contain the complete set of information on particular items).
Association: Security.
Source: NIST SP 800-53.

Recount: Process conducted for verifying the votes counted in an
election.
Association: Voting.
Source: No attribution.

Referendum: Contest between two (or more) choices in response to a
question (e.g., bond issue, recall, retention of a judge in office,
proposed amendment).
Association: Voting.
Source: NIST HF Rpt.

Repeatability: Ability to obtain independent test results by using
the same testing method on identical test items in the same testing
laboratory by the same operator using the same equipment within short
intervals of time.
Association: Testing, conformity assessment.
Source: ISO 5725.

Report: (n) Printed record, formatted for human readability, that
is produced by a voting system. (v) to produce such a record.
Association: Voting.
Source: No attribution.

Reproducibility: Ability to obtain test results with the same test
method on identical test items in different testing laboratories with
different operators using different equipment.

Association: Testing, conformity assessment.
Source: ISO 5725.

Requirement: Provision that conveys criteria to be fulfilled. See
also compliance point.
Association: Testing, standardization.
Source: NIST HB 150.

Residual Vote: Total number of votes that cannot be counted for a
specific contest. There may be multiple reasons for residual votes
(e.g., declining to vote for the contest, overvoting in a contest,
failure to cast ballot before leaving polling place).
Association: Voting, human factors.
Source: NIST HF Rpt.

Risk Assessment: Process of identifying the risks to system
security and determining the probability of occurrence, the resulting
impact, and additional safeguards that would mitigate this impact.
Association: Security.
Source: NIST SP 800-30.

Rolloff: Difference between number of votes cast for contests in
the higher offices on the ballot and the number cast for contests that
are lower on the ballot. It sometimes referred to as voter fatigue.
Association: Voting, human factors.
Source: NIST HF Rpt.

Runoff Election: Election to select a winner following a primary,
or sometimes a general election, in which no candidate in the contest
received the required minimum percentage of the votes cast. The two
candidates receiving the most votes for the race in question proceed to
the runoff election.
Association: Voting.
Source: VSS, IEEE 1583.
S
Second Chance Voting: Provides that voters are notified when their
ballots contain errors and are given a chance to correct them. Required
by HAVA 2002.
Association: Voting.
Source: NASS.

Secret Key: Cryptographic key that is used with a symmetric
cryptographic algorithm that is uniquely associated with one or more
entities and is not be made public. The use of the term ``secret'' in
this context does not imply a classification level, but rather implies
the need to protect the key from disclosure.
Association: Security.
Source: NIST SP 800-57.

Section 508: Amendment by Congress in 1998, to the Rehabilitation
Act to require federal agencies to make their electronic and
information technology accessible to people with disabilities. Section
508 was enacted to eliminate barriers in information technology.
Association: HF: accessibility
Source: No attribution.

Security Controls: Management, operational, and technical controls
(i.e., safeguards or countermeasures) prescribed for an information
system to protect the confidentiality, integrity, and availability of
the system and its information.
Association: Security.
Source: FIPS 199, NIST SP 800-53.

Semi-static Voting System Software: Software that contains
configuration information for the voting system based on the voting
equipment that is installed and the election being conducted. Semi-
static software is only modified during the installation of the voting
system software on voting equipment or the election specific software
such as ballot formats. See also voting system software.
Association: Voting.
Source: No attribution.

Specification, Technical: Document that prescribes technical
requirements to be fulfilled by a product, process or service.
Association: Standardization.
Source: ISO Guide 2-4.

Split Precinct: Precinct containing more than one ballot format in
order to accommodate a contiguous geographic area served by the
precinct that contains more than one election district.
Association: Voting.
Source: VSS, IEEE 1583.

Spoiled Ballot: Ballot that has been voted but will not be cast.
Association: Voting.
Source: No attribution.

Standard: Document established by consensus and approved by a
recognized body that provides, for common and repeated use, rules,
guidelines or characteristics for activities or their results, aimed at
the achievement of the optimum degree of order in a given context.
Association: Standardization.
Source: ISO Guide 2-4.

Standard, Product: Standard that specifies requirements to be
fulfilled by a product or a group of products, to establish its fitness
for purpose. A product standard may include, in addition to the fitness
for purpose requirements, directly or by reference, aspects such as
terminology, sampling, testing, packaging, and labeling and sometimes
processing requirements.
Association: Standardization.
Source: ISO Guide 2-6.

Standard, Testing: Standard that is concerned with test methods,
sometimes supplemented with other provision related to testing, such as
sampling, use of statistical methods, or sequence of test.
Association: Standardization.
Source: ISO Guide 2-6.

Standard on Data to Be Provided: Standard that contains a list of
characteristics for which values or other data are to be stated for
specifying the product, process, or service.
Association: Standardization.

[[Page 19005]]

Source: ISO Guide 2-4.

State Certification: State examination and possibly testing of a
voting system to determine its compliance with state laws, regulations,
and rules and any other state requirements for vote systems.
Association: Testing, conformity assessment, voting.
Source: VSS.

Static Voting System Software: Software that does not change based
on the election being conducted or the voting equipment upon which it
is installed, e.g., executable code. See also voting system software.
Association: Voting.
Source: No attribute.

Straight Party Voting: Mechanism by which voters are permitted to
cast a vote indicating the selection of all candidates on the ballot
for a single political party.
Association: Voting.
Source: VSS, IEEE 1583.

Support Software: Software that aids in the development or
maintenance of other software, for example, compilers, loaders and
other utilities.
Association: IT.
Source: VSS, IEEE 1583.

Symmetric (Secret) Encryption Algorithm: Encryption algorithms
using the same secret key for encryption and decryption.
Association: Security.
Source: NIST SP 800-49.
T
Tabulation: See Count.
Association: Voting.
Source: VSS, IEEE 1583.

T-Coil: Inductive coil used in some hearing aids to allow reception
of an audio band magnetic field signal, instead of an acoustic signal.
The magnetic or inductive mode of reception is commonly used in
conjunction with telephones, auditorium loop systems and other systems
that provide the required magnetic field output.
Association: Human Factors, HF: Accessibility.
Source: ANSI C63.19.

Tabulator: Device that counts votes.
Association: Voting.
Source: No attribution.

Technical Data Package: Vendor documentation relating to the voting
system that shall be submitted with the system as a precondition of
qualification testing.
Association: Testing, voting.
Source: VSS.

Telecommunications: Transmission, between or among points specified
by the user, of information of the user's choosing, without change in
the form or content of the information as sent and received.
Association: IT.
Source: IEEE 1583.

Test: Technical operation that consists of the determination of one
or more characteristics of a given product, process or service
according to a specified procedure.
Association: Testing.
Source: ISO Guide 2-4, NIST HB 150.

Test Campaign: Sum of the work by a VSTL on a single product or
system from contract through test plan, conduct of testing for each
requirement (including hardware, software, and systems), reporting,
archiving, and responding to issues afterwards.
Association: Testing, voting.
Source: NIST HB 150-22.

Test Case Specification: Document identifying the specific inputs
and expected result for each test identified in the test plan.
Association: Testing.
Source: IEEE 1583.

Test Design Specification: Expanded detail of the test approach
identified in the test plan for the related tests.
Association: Testing.
Source: IEEE 1583.

Test Method: Specified technical procedure for performing a test.
Association: Testing, conformity assessment.
Source: ISO Guide 2.

Test Plan: Document created prior to testing that outlines the
scope and nature of testing, items to be tested, test approach,
resources needed to perform testing, test tasks, risks and schedule.
Association: Testing, conformity assessment.
Source: IEEE 1583.

Testing: Determination of one or more characteristics of an object
of conformity assessment, according to a procedure. Testing typically
applies to materials, products, or processes.
Association: Testing, conformity assessment.
Source: ISO 17000.

Testing Authority: Organization that performs qualification testing
and produces qualification test reports. See also Voting System Testing
Laboratory.
Association: Testing, conformity assessment.
Source: No attribution.

Test Report for EAC Certification: Report of results of independent
testing of a voting system indicating the data testing was completed,
the specific system version tested, and the scope of tests conducted.
Association: Testing, voting.
Source: VSS, NIST HB 150.

Touch Screen Voting Machine: Machine that utilizes a computer
screen whereby a voter executes that voter's choices by touching
designated locations on the screen and that then tabulates those
choices.
Association: Voting.
Source: ME Statutes.

Traceability: Ability to relate a property of the result of a
measurement or the value of a standard to stated references.

Association: Testing.
Source: VIM.

Type font: Type of a given size and style, e.g., 10-point Bodoni
Modern.
Association: Human factors.
Source: ANSI Dict.

U
Uncertainty: Parameter, associated with the result of a measurement
that characterizes the dispersion of the values that could reasonably
be attributed to that which is being measured.
Association: Testing.
Source: VIM, NIST HB 150.

Undervote: (1) Occurs when the number of alternatives selected by a
voter in a contest is less than the maximum number allowed for that
contest. (2) Practice of voting for less than the total number of
election contests listed on the ballot, or of voting for less than the
number of positions to be filled for a single office (i.e., A person
would undervote if a contest required the selection of three out of a
given number of candidates, and the voter chose only two candidates).
Also known as undervoting.
Association: Voting.
Source: (1) NIST HF Rpt. (2) VSS, IEEE 1583, NASS.

Usability: Effectiveness, efficiency and satisfaction with which a
specified set of users can achieve a specified set of tasks in a
particular environment. Usability in the context of voting system
standards refers to voters being able to cast valid votes as they
intended quickly, without errors and with confidence that their ballot
choices as marked were recorded correctly. It also refers to the
usability of the setup of voting equipment for the election and the
running of the election by poll workers and election administrators.
Association: Human factors, HF: Usability.
Source: ISO 9241, NIST HF Rpt.

Usability Testing: Encompasses a range of methods that examine how
users in the target audience actually interact with a system, in
contrast to analytic techniques such as usability inspection.

[[Page 19006]]

Association: Human factors, HF: Usability.
Source: Usability First Usability Glossary.

User Documentation: See Voting Equipment User Documentation.
Association: Vote, test.
Source: IEEE 1583.
V
Valid Vote: Vote from a ballot or ballot image that conforms to
jurisdiction dependent criteria for accepting or rejecting entire
ballots, such as stray marks policies and voter eligibility criteria,
in a contest that was not overvoted.
Association: Voting.
Source: No attribution.

Validation: Process of evaluating a system or component during or
at the end of the development process to determine whether it satisfies
specified requirements.
Association: Testing.
Source: VSS.

Verification: Process of evaluating a system or component to
determine whether the products of a given development phase satisfy the
conditions (such as specifications) imposed at the start of the phase.
Association: Testing.
Source: VSS.

Verification and Validation (V&V): Process of determining whether
requirements for a system or component are complete and correct, the
products of each development phase fulfill the requirements or
conditions imposed by the previous phase, and the final system or
component complies with specified requirements.
Association: Testing.
Source: IEEE 1583.

Video Ballot: Electronic voter interface which presents ballot
information and voting instructions as video images. See also ballot.
Association: Voting, human factors, HF: Accessibility.
Source: FL Statutes.

Vote Capture Station: Component of a voting system that captures
and stores records of voter choices. See also witness device.
Association: Voting.
Source: No attribution.

Vote for N of M: Ballot choice in which voters are allowed to vote
for a limited number of candidates for a single office from a larger
field of candidates.
Association: Voting.
Source: VSS, IEEE 1583.

Voted Ballot: Ballot that a voter has finished filling in, but has
not yet cast or spoiled.
Association: Voting.
Source: No attribution.

Voter Registration System: Set of processing functions and data
storage that maintains records of eligible voters. This system
generally is not considered a part of a Voting System subject to the
2002 Voting System Standards.
Association: Voting.
Source: VSS.

Voter Verified Audit Record: (1) Human-readable printed record of
all of a voter's selections presented to the voter before the vote is
cast. (2) Printed version of the ballot that voters may view and check
for accuracy before their votes are cast. See also Voter Verified
Record or Voter Verified Paper Trail.
Association: Voting.
Source: (1) IEEE 1583, (2) NASS.

Voter-Verified Paper Trail (VVPT): See Voter Verified Audit Record.

Voting Environment: Aspects of the voting milieu outside of the
voting system that are encountered by voters, e.g., ramps, lighting,
noise, temperature, electro-magnetic interference. See also voting
equipment operational environment.
Association: Human factors, voting.
Source: No attribution.

Voting Equipment: Any mechanical, electromechanical, or electronic
components of a voting system. See also Electronic Voting Machine.
Association: Voting.
Source: No attribution.

Voting Equipment Operational Environment: All software, hardware
(including facilities, furnishings and fixtures), materials,
documentation, and the interface used by the election personnel,
maintenance operator, poll worker, and voter, required for voting
equipment operations. See also voting environment.
Association: Voting.
Source: IEEE 1583.

Voting Equipment Operations Procedures: Ordered steps that election
personnel, poll workers or voters follows to perform the tasks for each
operational environment.
Association: Voting.
Source: IEEE 1583.

Voting Equipment User Documentation: Electronic or printed material
that provides information for the election officials or voters.
Association: Voting.
Source: IEEE 1583.

Voting Machine: Mechanical or electronic equipment for the direct
recording and tabulation of votes. See also voting system.
Association: Voting.
Source: OH Statutes.

Voting Officials: Term used to designate the group of people
associated with elections, including election personnel, poll workers,
ballot designers and those responsible for the installation, operation
and maintenance of the voting systems.
Association: Voting.
Source: No attribution.

Voting Position: Specific response fields on a ballot where the
voter indicates the selection of a candidate or ballot proposition.
Association: Voting.
Source: VSS, IEEE 1583.

Voting Process: Entire array of procedures, people, resources,
equipment and locales by which elections are conducted.
Association: Voting.
Source: No attribution.

Voting Station: Location within the polling place where voters may
record their votes. A voting station includes the voting booth or
enclosure and the vote-recording device.
Association: Voting.
Source: VSS, IEEE 1583.

Voting System: Combination of mechanical, electromechanical, or
electronic equipment and any corresponding documentation. It includes
the software required to program, control, and support the equipment
that is used to define ballots; to cast and count votes; to report and/
or display election results; and to maintain and produce all audit
trail information. A voting system may also include the transmission of
results over telecommunication networks. It additionally includes the
associated documentation used to operate the system, maintain the
system, identify system components and their versions, test the system
during its development and maintenance, maintain records of system
errors and defects, and determine specific changes made after system
qualification. See also electronic voting machine, voting equipment,
voting machine.
Association: Voting.
Source: VSS.

Voting System Software: All the executable code and associated
configuration files needed for the proper operation of the voting
system regardless of the location of installation and functionality
provided. This includes third party software such as operating systems,
drivers, etc. See also dynamic voting system software, semi-static
voting system software, and static voting system software.
Association: Voting.
Source: No attribution.

[[Page 19007]]

Voting System Testing: Examination and testing of a computerized
voting system by using test methods to determine if the system complies
with the requirements in the Voluntary Voting System Guidelines and
with its own specifications. This process occurs prior to EAC
certification and subsequent State certification.
Association: Testing, voting.
Source: VSS.

Voting System Testing Laboratory (VSTL): Testing laboratory
accredited by the National Voluntary Laboratory Accreditation Program
for testing of voting systems. The Director of NIST submits a list of
independent, non-Federal VSTLs to the EAC for accreditation.
Association: Testing.
Source: NIST HB 150-22.

VVPAT-Ballot Box: Ballot box containing the paper record.
Association: Security, voting.
Source: No attribution.

VVPAT-Display: Transparent covering over the paper record printed
by the DRE-VVPAT. It permits a voter to inspect the paper record but
prevents the voter from physically handling the paper record.
Association: Security, voting.
Source: No attribution.

VVPAT-Printer: Printing capability of the voting system, including
the printer and any associated device involved in printing the paper
records and transferring them to ballot boxes.
Association: Security, voting.
Source: No attribution.
W
Witness Device: Component of a voting system that captures voter
verification of the records at the voting station. See also vote
capture station.
Association: Voting.
Source: No attribution.

Write-in Voting: Means to cast a vote for an individual not listed
on the ballot. Voters may do this by using a marking device to
physically write their choice on the ballot or they may use a keypad,
touch screen or other electronic means to indicate their choice.
Association: Voting.
Source: VSS, IEEE 1583.

Workspace: See voting station.
Association: Voting.
Source: VSS.

A.2 Sources

Definitions in this Glossary are either extracted from or based on
the following sources:
44 U.S.C. United States Code, Title 44, Chapter 35, Information
Security, Section 3542, Definitions.
ACM SIGCHI ACM's Special Interest Group on Computer-Human Interaction,
http://www.acm.org/sigchi/ (February 2005).

ADA Americans with Disabilities Act of 1990.
ANSI Dict. American National Dictionary for Information Processing
Systems, American National Standards Committee X3, Information
Processing Systems, 1982.
ANSI 354 American National Standards Institute, InterNational Committee
for Information Technology Standards, Common Industry Format for
Usability Test Reports, ANSI/INCITS 354-2001
ANSI C63.19 American National Standards for Methods of Measurement of
Compatibility between Wireless Communications Devices and Hearing Aids,
2001. electionline electionline.org/, (March 2005).
FIPS 140-2 Federal Information Processing Standard 140-2, Security
Requirements for Cryptographic Modules, May 2001.
FIPS 199 Federal Information Processing Standard 199, Standards for
Security Categorization of Federal Information and Information Systems,
December 2003.
FIPS 201 Federal Information Processing Standard 201, Personal Identity
Verification for Federal Employees and Contractors, February 2005.
FL Statutes Florida Statutes: Section 97.021(3) and Section
101.56062(1)(n) Standards for accessible voting.
HAVA Help America Vote Act of 2002--Public Law 107-252.
IEA International Ergonomics Association, http://www.iea.cc/, (February

2005).
IEEE 1583 IEEE P1583/D5.3.2 Draft Standard for the Evaluation of Voting
Equipment, December 6, 2004.
IL Statutes Illinois Public Act 093-0574.
ISO 5725 ISO/IEC 5725:1994 Accuracy (trueness and precision) of
measurement methods and results.
ISO 9241 ISO/IEC 9241:1997 Ergonomic requirements for office work with
visual display terminals (VDT).
ISO 17000 ISO/IEC 17000:2004 Conformity assessment--Vocabulary and
general principles.
ISO Guide 2-4 ISO/IEC Guide 2:2004 Standardization and related
activities--General vocabulary.
ISO Guide 2-6 ISO/IEC Guide 2:1996 Standardization and related
activities--General vocabulary.
ME Statutes Maine LD 1759 Enacted 4/22/2004.
NASS National Association of Secretaries of State Election Reform Key
Terms, http://www. nass.org/Election% 20Reform% 20Key% 20Terms.pdf

(February 2005).
NIST HB 143 NIST Handbook 143 State Weights and Measures Laboratories
Program Handbook.
NIST HB 150 NIST Handbook 150:2001 NVLAP Procedures and General
Requirements.
NIST HF Rpt. NIST Special Publication 500-256 Improving the Usability
and Accessibility of Voting Systems and Products, May 2004.
NIST SP 800-30 NIST Special Publication 800-30 Risk Management Guide
for Information Technology Systems, July 2002.
NIST SP 800-49 NIST Special Publication 800-49 Federal S/MIME V3 Client
Profile, November 2002.
NIST SP 800-53 NIST Special Publication 800-53 Recommended Security
Controls for Federal Information Systems, Appendix B, Glossary.
NIST SP 800-59 NIST Special Publication 800-59 Guideline for
Identifying an Information System as a National Security System, August
2003.
NIST SP 800-63 NIST Special Publication 800-63 Electronic
Authentication Guideline: Recommendations of the National Institute of
Standards and Technology, June 2004.
OH Statutes Ohio HB-262 enacted 5/7/2004.
OMB A130 OMB Circular A-130, Appendix III.
Section 508 Electronic and Information Technology Accessibility
Standards (2002) Architectural and Transportation Barriers Compliance
Board, 36 CRF Part 1194, http://www. accessboard.gov/ sec508/

508standards. htm.
Usability Glossary Usability First Usability Glossary, http://www.

usabilityfirst. com/glossary/ main.cgi, (February 2005).
VIM The ISO International Vocabulary of Basic and General Terms in
Metrology (VIM), 1994.
VSS Voting Systems Standards of 2002 (Federal Election Commission),
Volumes I and II.
Whatis.com Whatis.com, IT Encyclopedia, http://whatis. techtarget. com/

definition/ 0,,sid9-- gci491925,00. html (February 2005).
WordNet WordNet [reg]2.0, (copyright) 2003 Princeton University.

A.3 List of Associations

Conformity Assessment
Human Factors (HF)

[[Page 19008]]

HF: accessibility
HF: usability
IT--Information Technology
Security
Software Engineering
Standardization
Testing
Typography
Voting

A.4 List of Deprecated Terms

The following terms are being phased out and replaced by newer
terms. Note that there is a transition period where both terms are in
use at the same time.

------------------------------------------------------------------------
Deprecated term Replaced by
------------------------------------------------------------------------
Certification Testing..................... State Certification
Electronic Cast Vote Record............... Cast Vote Record
Qualification Number...................... no replacement at this time
Qualification Test Report................. Test Report for EAC
Certification
Qualification Testing..................... Voting System Testing
------------------------------------------------------------------------

Volume I, Appendix B

Table of Contents

B Appendix--Applicable Documents

B.1 Documents Incorporated in the Standards
B.2 Standards Development Documents
B.3 Guidance Documents

B Appendix--Applicable Documents

B.1 Documents Incorporated in the Standards

The following publications have been incorporated into the
Standards. When specific provisions from these publications have been
incorporated, specific references are made in the body of the
Standards.

------------------------------------------------------------------------

------------------------------------------------------------------------
Federal Regulations......... Code of Federal Regulations, Title 20,
Part 1910, Occupational Safety and Health
Act.
Code of Federal Regulations, Title 36,
Part 1194, Architectural and
Transportation Barriers Compliance Board,
Electronic and Information Technology
Standards--Final Rule.
Code of Federal Regulations, Title 47,
Parts 15 and 18, Rules and Regulations of
the Federal Communications Commission.
Code of Federal Regulations, Title 47,
Part 15, ``Radio Frequency Devices'',
Subpart J, ``Computing Devices'', Rules
and Regulations of the Federal
Communications Commission.
American National Standards ANSI C63.4.......... Methods of
Institute (ANSI). Measurement of
Radio-Noise
Emissions from Low-
Voltage Electrical
and Electronic
Equipment in the
Range of 9Khz to 40
GHz.
ANSI C63.19......... American National
Standard for
Methods of
Measurement of
Compatibility
between Wireless
Communication
Devices and Hearing
Aids.
ANSI-NCITS 354-2001. Industry Usability
Reporting and the
Common Industry
Format.
International IEC 61000-4-2 (1995- Electromagnetic
Electrotechnical Commission 01). Compatibility (EMC)
(IEC). Part 4: Testing and
Measurement
Techniques. Section
2 Electrostatic
Discharge Immunity
Test (Basic EMC
publication).
IEC 61000-4-3 (1996) Electromagnetic
Compatibility (EMC)
Part 4: Testing and
Measurement
Techniques. Section
3 Radiated Radio-
Frequency
Electromagnetic
Field Immunity
Test.
IEC 61000-4-4 (1995- Electromagnetic
01). Compatibility (EMC)
Part 4: Testing and
Measurement
Techniques. Section
4 Electrical Fast
Transient/Burst
Immunity Test.
IEC 61000-4-5 (1995- Electromagnetic
02). Compatibility (EMC)
Part 4: Testing and
Measurement
Techniques. Section
5 Surge Immunity
Test.
IEC 61000-4-6 (1996- Electromagnetic
04). Compatibility (EMC)
Part 4: Testing and
Measurement
Techniques. Section
6 Immunity to
Conducted
Disturbances
Induced by Radio-
Frequency Fields.
IEC 61000-4-8 (1993- Electromagnetic
06). Compatibility (EMC)
Part 4: Testing and
Measurement
Techniques. Section
8 Power-Frequency
Magnetic Field
Immunity Test.
(Basic EMC
publication).
IEC 61000-4-11 (1994- Electromagnetic
06). Compatibility (EMC)
Part 4: Testing and
Measurement
Techniques. Section
11. Voltage Dips,
Short Interruptions
and Voltage
Variations Immunity
Tests.
IEC 61000-5-7 Ed. Electromagnetic
1.0 b:2001. compatibility (EMC)
Part 5-7:
Installation and
mitigation
guidelines-Degrees
of protection
provided by
enclosures against
electromagnetic
disturbances.
National Institute of FIPS 140-2.......... Security
Standards and Technology. Requirements for
Cryptographic
Modules.
FIPS 180-2.......... Secure Hash
Standard, August
2002.
FIPS 186-2.......... Digital Signature
Standard, February
2000.
FIPS 188............ Standard Security
Label for
Information
Transfer.
FIPS 196............ Entity
Authentication
Using Public Key
Cryptography.

[[Page 19009]]

FIPS 197............ Advanced Encryption
Standard (AES).
SP 800-63........... Electronic
Authentication
Guideline, Version
1.0.1.
Military Standards.......... MIL-STD-498......... Software Development
and Documentation
Standard, 1989.
MIL-STD-810D (2).... Environmental Test
Methods and
Engineering
Guidelines, 19 July
1983.
------------------------------------------------------------------------

B.2 Standards Development Documents

The following publications have been used for guidance in the
revision of the Standards.

------------------------------------------------------------------------

------------------------------------------------------------------------
American National Standards ANSI/ISO/IEC TR Information
Institute (ANSI). 9294.1990. Technology
Guidelines for the
Management of
Software
Documentation.
ISO/IEC TR 13335- Information
4:2000. technology--Guideli
nes for the
management of IT
Security--Part 4:
Selection of
safeguards.
International Organization ISO/IEC TR 13335- Information
for Standardization (ISO). 3:1998. technology--Guideli
nes for the
management of IT
Security--Part 3
Techniques for the
management of IT
security.
ISO/IEC TR 13335- Information
2:1997. technology--Guideli
nes for the
management of IT
Security--Part 2:
Managing and
planning IT
security.
International Electro- ISO/IEC TR 13335- Information
technical Commission (IEC). 1:1996. technology--Guideli
nes for the
management of IT
Security--Part 1:
Concepts and models
for IT security.
ISO 10007:1995...... Quality Mgmt.
Guidelines for
Configuration
Management.
ISO 10005-1995...... Quality Mgmt.
Guidelines for
Quality Plans.
ANSI/ISO/ASQC QS9000- QM and QA standards
3-1997. Part 3: Guidelines
for the application
of ANSI/ISO/ASQC
Q9000-1994 to the
Development,
Supply,
Installation, and
Maintenance of
Computer Software.
Electronic Industries MB2, MB5, MB9....... Maintainability
Alliance Standards. Bulletins.
EIA 157............. Quality Bulletin.
EIA QB2-QB5......... Quality Bulletins.
EIA RB9............. Failure Mode and
Effect Analysis,
Revision 71.
EIA SEB1-SEB4....... Safety Engineering
Bulletins.
RS-232-C............ Interface Between
Data Terminal
Equipment and Data
Communications
Equipment Employing
Serial Binary Data
Interchange.
RS-366-A............ Interface Between
Data Terminal
Equipment and
Automatic Calling
Equipment for Data
Communication.
RS-404.............. Standard for Start-
Stop Signal Quality
Between Data
Terminal Equipment
and Non-synchronous
Data Communication
Equipment.
National Institute of NISTIR 4909......... Software Quality
Standards and Technology. Assurance:
Documentation and
Reviews.
Institute of Electrical and 610.12-1990......... IEEE Standard
Electronics Engineers. Glossary of
Software
Engineering
Terminology
730-1998............ IEEE Standard for
Software Quality
Assurance Plans
828-1998............ IEEE Standard for
Software
Configuration
Management Plans
829-1998............ IEEE Standard for
Software Test
Documentation
830-1998............ IEEE Recommended
Practice for
Software
Requirements
Specifications.
Military Standards.......... MIL-STD-498......... Software Development
and Documentation,
27 May 1998.
------------------------------------------------------------------------

B.3 Guidance Documents

The following publications contain information that is useful in
understanding and complying with the Standards.

[[Page 19010]]

------------------------------------------------------------------------

------------------------------------------------------------------------
American National Standards ANSI/ISO/IEC TR Information
Institute (ANSI). 10176.1998. Technology
Guidelines for the
Preparation of
Programming
Language Standards.
ANSI/ISO/IEC Information
6592.2000. Technology
Guidelines for the
Documentation of
Computer Based
Application
Systems.
International Organization ANSI/ISO/ASQC Q9000- Quality management
for Standardization (ISO). 3-1997. and quality
assurance standards
Part 3: Guidelines
for the application
of ANSI/IAO/ASQC
Q9001-1994 to the
Development,
supply,
installation and
maintenance of
computer software.
International Electro- ANSI/ISO/ASQC Q9000- Quality Management
technical Commission (IEC). 1-1994. and Quality
Assurance
Standards--Guidelin
es for Selection
and Use.
ANSI/ISO/ASQC Q10007- Quality Management
1995. Guidelines for
Configuration
Management.
ANSI X9.31-1998..... Digital Signatures
Using Reversible
Public Key
Cryptography for
the Financial
Services Industry,
1998.
ANSI X9.62-1998..... Public Key
Cryptography for
Financial Services
Industry: The
Elliptic Curve
Digital Signature
Algorithm, 1998.
ISO/IEC 9594-8:2001. ITU-T Recommendation
X.509 (2000),
Information
technology--Open
Systems
Interconnection--Th
e Directory: Public-
key and attribute
certificate
frameworks.
National Institute of FIPS 102............ Guideline for
Standards and Technology. Computer Security
Certification and
Accreditation.
FIPS 112............ Password Usage (3).
FIPS 113............ Computer Data
Authentication.
Institute of Electrical and 488-1987............ IEEE Standard
Electronics Engineers. Digital Interface
for Programmable
Instrumentation.
796-1983............ IEEE Standard
Microcomputer
System Bus IEEE/
ANSI Software
Engineering
Standards.
750.1-1995.......... IEEE Guide for
Software Quality
Assurance Planning.
1008-1987........... IEEE Standard for
Software Unit
Testing.
1016-1998........... IEEE Recommended
Practice for
Software Design
Descriptions.
1012-1998........... IEEE Guide for
Software
Verification and
Validation Plans.
Military Standards.......... MIL-HDBK-454........ Standard General
Requirements for
Electronic
Equipment.
MIL-HDBK-470........ Maintainability
Program for Systems
& Equipment.
MIL-HDBK-781A....... Handbook for
Reliability Test
Methods, Plans, and
Environments for
Engineering,
Development
Qualification, and
Production.
MIL-STD-882......... Systems Safety
Program
Requirements.
MIL-STD-1472........ Human Engineering
Design Criteria for
Military Systems,
Equipment and
Facilities.
MIL-STD-973......... Configuration
Management, 30
September 2000.

[[Page 19011]]

Other References............ .................... Designing for the
Color-Challenged: A
Challenge, by
Thomas G. Wolfmaier
(March 1999); http://www.sandia.gov/
newsletter/

mar99/
accessibility--colo
r--challenged.html;
Effective Color
Contrast: Designing
for People with
Partial Sight and
Color Deficiencies,
by Aries Arditi,
Ph.D; http://www.lighthouse.org/
Markup

Language (EML),
Version 4.0,
(Committee Draft)
Organization for
the Advancement of
Structured
Information
Standards (OASIS),
January 24, 2005.
RSA Laboratories
Technical Note,
Public Key
Cryptographic
Standard (PKCS)
7:
Cryptographic
Message Syntax
Standard, November
1, 1993.
RSA Laboratories
Technical Note,
Extensions and
Revisions to PKCS
7, May 13,
1997.
The Americans with
Disabilities Act
Accessibility
Guidelines (ADAAG
2202), Access
Board; http://www.access-
.

------------------------------------------------------------------------

Volume I, Appendix C

Table of Contents

C Appendix--Best Practices for Voting Officials
C.1 Best Practices for Human Factors
C.2 Best Practices for Security

Best Practices for Voting Officials

Many requirements for human factors and security (e.g., wireless
communications, software distribution, and setup validation, voter
verified paper audit trails) depend not only on voting systems
providing specific capabilities but on voting officials developing and
carrying out appropriate procedures. Consequently, the Voluntary Voting
System Guidelines (VVSG) Version 1 provides guidance in the form of
best practices for voting officials. These best practices provide
adjuncts to the technical requirements for voting systems in order to
ensure the integrity of the voting process and to assist States in
properly setting up, deploying, and operating voting systems.
This appendix contains a list of best practices that have been
extracted from the body of the VVSG Version 1. The section numbering
and introductory text from the VVSG has been retained to provide the
context for the best practice as well as to indicate from where it was
extracted.

C.1 Best Practices for Human Factors

2.2.7 Human Factors
Human factors is concerned with the understanding of interactions
among humans and other elements of a system. The importance of human
factors in the design of voting systems has become increasingly
apparent. It is not sufficient that the internal operation of these
systems is correct; in addition, voters and poll workers must be able
to use them effectively. The challenge, then, is to provide a voting
system and voting environment that all voters can use comfortably,
efficiently, and with justified confidence that they have cast their
votes correctly.
2.2.7.1 Accessibility
The Help America Vote Act (HAVA) Section 301(a)(3) reads in part:
``Accessibility for individuals with disabilities--The voting system
shall: (A) be accessible for individuals with disabilities, including
nonvisual accessibility for the blind and visually impaired, in a
manner that provides the same opportunity for access and participation
(including privacy and independence) as for other voters; (B) satisfy
the requirement of subparagraph (A) through the use of at least one
direct recording electronic voting system or other voting system
equipped for individuals with disabilities at each polling place.''
Ideally every voter would be able to vote independently and
privately.
Best Practices
When the provision of accessibility involves an
alternative format for ballot presentation, then all the other
information presented to voters in the case of non-disabled English-
literate voters (including instructions, warnings, messages, and ballot
choices) is also presented in that alternative format.
When the primary means of voter identification or
authentication uses biometric measures that require a voter to possess
particular biological characteristics, the voting process provides a
secondary means that does not depend on those characteristics.
Polling places are subject to the appropriate guidelines
of the Americans with Disabilities Act (ADA) of 1990 and of the
Architectural Barriers Act (ABA) of 1968.
On all voting stations, the default color coding maximizes
correct perception by voters and operators with color blindness.
A sanitized headphone or handset is made available to each
voter.
If the normal procedure is for voters to submit their own
ballots, then the voting process provides features that enable voters
who are blind to perform this submission.
The Acc-VS provides a clear floor space of 30 inches (760
mm) minimum by 48 inches (1220 mm) minimum for a stationary mobility
aid. The clear floor space is level with no slope exceeding 1:48 and
positioned for a forward approach or a parallel approach.
All controls, keys, audio jacks and any other part of the
Acc-VS necessary for the voter to operate the voting system are within
the reach regions as specified in the VVSG Volume I, Section
2.2.7.1.4.3.
The Acc-VS incorporates the features listed in the VVSG
Volume I, Section 2.2.7.1.2.2.3 (audio presentation) to provide
accessibility to voters with hearing disabilities.
The voting process is made accessible to voters with
cognitive disabilities.
2.2.7.2 Limited English Proficiency
HAVA Section 301(a)(4) reads in part:
``Alternative language accessibility--The voting system shall
provide alternative language accessibility pursuant to the requirements
of section 203 of the Voting Rights Act of 1965 (42 U.S.C. 1973aa-
1a).''
Ideally every voter would be able to vote independently and
privately, regardless of language.

[[Page 19012]]

Best Practices
Regardless of the language, candidate names are displayed
or pronounced in English on all ballots. For written languages that do
not use Roman characters (e.g. Chinese, Japanese, Korean, Arabic), the
ballot includes transliteration of candidate names into the relevant
language.
2.2.7.3 Usability
HAVA Section 301 begins by addressing the interaction between the
voter and the voting system. In addition to these mandates, HAVA
Sections 243 and 221(e)(2)(D) address support for improved usability.
Usability is defined generally as a measure of the effectiveness,
efficiency, and satisfaction achieved by a specified set of users with
a given product in the performance of specified tasks. In the context
of voting, the primary users are the voters (but also poll workers),
the product is the voting system, and the task is the correct
representation of one's choices in the election.
Best Practices
The voting station does not visually present a single race
spread over two pages or two columns.
The ballot clearly indicates the maximum number of
candidates for which one can vote within a single race.
The ballot presents the relationship between the name of a
candidate and the mechanism used to vote for that candidate in a
consistent manner.
2.2.7.4 Privacy
Voter privacy is strongly supported by HAVA--Sections 221(e)(2)(C)
and 301(a)(1). Privacy in the voting context, including the property of
the voter being unable to disclose his or her vote, ensures that the
voter can make choices based solely on his or her own preferences
without intimidation or inhibition. Among other practices, this forbids
the issuance of a receipt to the voter that would provide proof to
another how he or she voted.
Note that these best practices address privacy concerns in relation
to human factors issues and not with respect to the processing of cast
ballots.
Best Practices
The ballot and any input controls are visible only to the
voter during the voting session and ballot submission. Poll workers
need to take into account such factors as visual barriers, windows,
permitted waiting areas for other voters, and procedures for ballot
submission when not performed at the voting station, e.g. submission of
optiscan ballots to a central reader.
The audio interface is audible only to the voter.
As mandated by HAVA 301(a)(1)(C), the voting system
notifies the voter of an attempted overvote in a way that preserves the
privacy of the voter and the confidentiality of the ballot.
Appropriate procedures are needed to ensure that absentee
balloting enable the voter to preserve privacy. There is no practical
means to prevent a voter from revealing an absentee paper ballot to
others. But the procedures should ensure that if a voter chooses to
maintain privacy, it is not violated at a later stage, in particular
when the ballot is received by voting officials.

C.2 Best Practices for Security

VVSG Version 1 addresses four new aspects of voting systems
security. The first, independent dual verification is informative and
provide characteristics of these systems. It does not yet contain any
best practices. There are best practices for the other three sections:
Voter Verified Paper Audit Trails, Wireless Requirements, and Software
Distribution and Setup Validation.
6.0.2 Requirements for Voter Verified Paper Audit Trails
VVSG Version 1 provides requirements for voter verified paper audit
trails (VVPAT) so that States that choose to implement VVPAT or States
that are considering implementation can utilize these requirements to
help ensure the effective operation of these systems.
6.0.2.4 Approve or Spoil the Paper Record
Best Practices
Appropriate procedures are needed for reconciling the
number of spoiled paper records with the number of spoiled electronic
records and for addressing any discrepancies after the close of polls.
Appropriate procedures are needed to permit the voter to
cast a ballot if the maximum number of spoiled ballots occurs.
Appropriate procedures are needed to address situations in
which a voter is unable to review the paper record.
Appropriate procedures are needed to address situations in
which a voter indicates that the electronic and paper records do not
match. If the records do not match, a potentially serious error has
occurred. Election officials should first verify that the records do
not match and then take appropriate actions such as removing the voting
station from service and quarantining its records for later analysis.
6.0.2.5 Preserve Voter Privacy and Anonymity
Best Practices
Appropriate procedures are needed to ensure the privacy
and anonymity of voters whose paper records contain any of the
alternative languages chosen for making ballot selections.
Appropriate procedures are needed to prevent voters from
leaving the voting area with a paper record that can directly reveal
the voter's choices.
6.0.2.7 Equipment Security, Reliability, and Maintainability
Best Practices
Appropriate procedures are needed to ensure that voting
systems are physically secured from tampering and intentional damage.
6.0.3 Wireless Requirements
Wireless is defined as any means of communication that occurs
without wires. This includes radio frequency (RF), infrared, (IR) and
microwave. The use of wireless technology within a voting system
introduces risk and should be approached with caution. Wireless
communication is susceptible to disruption, eavesdropping, and
interference from other wireless signals. The combination of technical
features and functionality built into the voting system along with
procedural practices in using and handling the voting system can
mitigate the risks of using wireless communications.
6.0.3.2 Controlling Usage
Best Practices
When using encryption to ensure that the wireless
communication is secure, appropriate procedures are needed for
cryptographic key management.
6.0.3.6 Protecting the Voting System From a Wireless-Based Attack
Best Practices
Appropriate procedures are needed to ensure that wireless
communication actions are logged and capture at least the following
information: times wireless is activated and deactivated, services
accessed, identification of device to which data was transmitted to or
received from, identification of authorized user, successful and
unsuccessful attempts to access wireless communications or service.
6.0.4 Distribution of Voting System Software and Setup Validation
The goal of software distribution requirements is to ensure that
the correct voting system software has been

[[Page 19013]]

distributed without modification. The goal of setup validation
requirements, including requirements for verifying the presence of
qualified software and the absence of other software, is to ensure that
voting system equipments is in a proper initial state before being
used.
6.0.4.1 Software Distribution Methodology Requirements
Best Practices
Voting software used to install the qualified voting
systems can be obtained on write-once media from the voting system
vendor or an EAC accredited test authority.
The reference information produced by the NSRL or other
EAC designated repository can be used to verify that the correct
software has been received.
6.0.4.2 Generation and Distribution Requirements for Reference
Information
Best Practices
To ensure that the write-once media contains the correct
information, a digital signature can be used. The digital signature can
replace secure storage of reference information since the digital
signature can be used to verify that the reference information media
has not been modified or corrupted.
The vendor's documented values can be used to verify that
all voting systems' static and initial register and variable values are
correct prior to an election.
The reference information can be used to verify that
voting system software is the correct version of the software prior to
an election.
If differences between the reference information and
voting system software are found, then appropriate procedures are
needed to handle and resolve these anomalies.

Volume I, Appendix D

Table of Contents

D Appendix--Independent Dual Verification (Informative)

D.1 Independent Dual Verification Systems
D.2 Core Characteristics for IDV Systems
D.3 Split Process IDV Systems
D.4 Witness IDV Systems
D.5 End to End (Cryptographic) IDV Systems

Appendix D

Appendix D is an informative section with characteristics of
independent dual verification systems followed by characteristics of
the types of independent dual verification systems which will be
used as the basis for future requirements. They are preliminary and
will be evolving with further research.

D.1. Independent Dual Verification Systems

A primary objective for using electronic voting systems is the
production of voting records that are highly precise, highly
reliable, and easily counted--in essence, an accurate representation
of ballot choices whose handling requirements are reasonable. To
meet this objective, there are many factors to consider in an
electronic voting system's design, including:
The environment provided for voting, including the
voting site and various environmental factors,
The ease with which voters can use the voting system,
i.e., its usability,
The robustness and reliability of the voting equipment,
and
The capability of the records to be used in audits.
Independent Dual Verification (IDV) systems have as their
primary objective the production of ballot records that are capable
of being used in audits in which their correctness can be audited to
very high levels of precision. The primary security issues addressed
by IDV systems are:
Whether electronic voting systems are accurately
recording ballot choices, and
Whether the ballot record contents can be audited
precisely post-election.
The threats addressed by IDV systems are those that could cause
a voting system to inaccurately record the voter's intent or cause a
voting system's records to become damaged, i.e., inserted, deleted,
or changed. These threats could occur via any number of means
including accidental damage or various forms of fraud. The threats
are addressed mainly by providing, in the voting system design, the
capability for ballot record audits to detect precisely whether
specific records are correct as recorded or damaged, missing, or
fraudulent.

1.1 Independent Dual Verification Systems: Improved Accuracy in
Audits

Independent Verification is the top-level categorization for
electronic voting systems that produce multiple records of ballot
choices whose contents are capable of being audited to high levels
of precision. For this to happen, the records must be produced and
made verifiable by the voter, and then subsequently handled
according to the following protocol:
At least two records of the voter's choices are
produced and one of the records is then stored such that it cannot
be modified by the voting system, e.g., the voting system creates a
record of the voter's choices and then copies it to some write-once
media.
The voter must be able to verify that both records are
correct, e.g., verify his or her choices on the voting system's
display and also verify the second record of choices stored on the
write-once media.
The verification processes for the two verifications
must be independent of each other and (a) at least one of the
records must be verified directly by the voter, or (b) it is
acceptable for the voter to indirectly verify both records if they
are stored on different systems produced by different vendors.
The content of the two records can be checked later for
consistency through the use of identifiers that allow the records to
be linked.
An assumption is made that at least one set of records is usable
in an efficient counting process such as by using an electronic
voting system, and the other set of records is usable in an
efficient process of verifying its agreement with the other set of
records used in the counting process. The sets of records would
preferentially be different in form and thus have more resistance to
accidental or deliberate damage.
Given these conditions above, the multiple records are said to
be distinct and independently verifiable, that is, both records are
not under the control of the same processes. As a result of this
independence, one record can be used to audit or check up on the
accuracy of the other record. Because the storage of the records is
separate, an attacker who can compromise one of the records still
will face a difficult task in compromising the other.

1.2 Example Independent Dual Verification Systems

The following sections present overviews of several types of IDV
systems. Some of these systems have not been marketed as yet but are
included here to help clarify approaches to independent verification
systems. The systems discussed are:
Voting systems with a split process architecture,\1\
---------------------------------------------------------------------------

\1\ The split process architecture is otherwise known as the
frog protocol, which was first described in the Caltech-MIT report:
Voting: What is, What Could Be, as part of a modular voting
architecture. The frog term, i.e., the token, was chosen
specifically to convey no information about the physical form of the
object used to carry vote information between two separate modules
of the voting station. The report is available for download at
http://www.vote.caltech.edu/.

---------------------------------------------------------------------------

End-to-end voting systems that include cryptographic
audit schemes,
Witness voting systems that take a picture of or
otherwise capture an indirect verification of ballot choices, and
Direct independent verification, including some types
of voting systems that produce an optically scanned ballot or that
produce a voter-verified paper audit trail (VVPAT).

1.2.1 The Split Process Architecture for IDV Systems

A voting machine with a split process architecture consists of
vote capture and verification stations that are separate, i.e., two
physical devices. A voter inserts an object called a token into the
capture station to make ballot selections and then takes the token
object to the verification station to review and store his or her
votes. The token object could be paper or some write-once read-only
media. Two records of the vote are created: One on the token object
and one by the verification station. Either could be used in the
final count.
For any split process voting system, the interaction between the
voter and the split process operates as follows:
1. A voter is given a token object that has been initialized to
be blank.
2. Supporting information is written to the token object
including the ballot and identification information about the
election and precinct.

[[Page 19014]]

3. The voter inserts the token object into a capture station
such as a DRE, which reads the ballot information from the token and
then displays the ballot on an input device such as a touch screen.
The voter to makes his or her ballot choices, which causes a record
of the vote to be recorded on the token object.
4. The voter takes the token object to a separate verification
station, which reads the recorded votes from the token object, makes
an electronic copy, and displays it to the voter.
5. The voter verifies that the information is correct and then
deposits the token object into a container where it can be archived
and used later for recounts or audits against the electronic
records.
Two sets of records are produced: The electronic records and the
token's records. Typically, the electronic records recorded by the
verification station would be counted in the election. At least one
of the sets of records should be different in form from the other
set of records and be resistance to accidental or deliberate damage
so that it can remain useful for audits and recounts.
In theory, the physical separation of the ballot capture from
the ballot verification may make analysis of the capture and
verification devices easier or less costly. The rationale is that
the user interface software on the capture station is expected to be
complex and difficult to verify for correctness. On the other hand,
the verification station's software is expected to be less
complicated because it need only copy the contents of the token,
display it to the voter, and store the ballot choices.
The verification station's software is considered to be the
``trusted computing base'' of the voting system, because it must be
trusted in the verification process and then trusted to store the
record for counting, i.e, cast the voter's ballot. The software to
implement this capability should be relatively small and thus easier
to inspect and test.
In general, segregating functions by placing them on physically
different systems is a standard computer security practice for
making those functions easier to test for correctness and easier to
manage securely.

1.2.2 End to End (Cryptographic) IDV Systems

End to end voting systems use cryptographic techniques to store
an encrypted copy of the voter's ballot choices. In this way,
ballots can be audited and demonstrated to have been included in the
election count.
End to end systems in existence today generally operate as
follows:
1. A voter uses a voting station such as a DRE to make ballot
choices.
2. The DRE issues a paper receipt to the voter that contains
information that permits the voter to verify that the choices were
recorded correctly. The information does not permit the voter to
reveal his or her choices.
3. The voter may have the option to check that his or her ballot
choices were included in the election count, e.g., by checking a Web
site of values that (should) match the information on the voter's
paper receipt.
End to end systems are sometimes referred to as receipt-based
systems. They may provide an assurance not only that the correct set
of ballot choices was recorded, but that those choices were included
in the election count. Some analyses of auditing and cryptographic
systems assert that very small numbers of self-audits are required
to verify the correctness of an election.

1.2.3 Witness IDV Systems

A witness voting system creates the second record of ballot
choices by using a separate module to record or witness the voter's
verification of the first record. The primary feature of a witness
system is that the creation of the record does not require action by
the voter. This may result in quicker voting times or voting systems
that are simpler to use than other approaches that involve multiple,
direct verifications by the voter.
An example of a witness system is a DRE with a camera mounted
above its screen. The camera takes pictures and saves them
independently of the DRE. It would operate as follows:
1. A voter makes ballot choices at the DRE and then presses a
button to record his or her vote.
2. The DRE records the ballot choices and uses them in the
election count.
3. At the time the button is pressed, the camera takes a picture
of the DRE's screen and saves the image (the voter is not included
in the picture).
4. This collection of images constitutes a second ballot record
that can be used in audits and recounts.
As can be seen by this example, the voter's interactions are
reduced to making ballot choices at the DRE and pressing a button to
make the selections final. If the DRE were to be compromised such
that it secretly recorded the ballot choices incorrectly, the stored
photographic images would reflect what the voter had seen and
verified at the DRE's screen.
Because the voter may not be able to verify that the creation of
the second record was performed accurately, it is important that the
creation process be highly reliable and very resistant to accidental
or deliberate damage. Also, the suitability of the records for
manual or automated auditing is a factor when considering this
approach.

1.2.4 Direct IDV Systems

Direct independent dual verification systems produce a record
for voter verification that the voter may verify directly with the
voter's senses and which is then preserved for auditing or counting.
Some optical scan voting system approaches fit into this category
(albeit loosely), as well as those systems with VVPAT (Voter
Verified Paper Audit Trail) capability.
Some optical scan voting system approaches fit into this
category (albeit loosely), as well as those systems with VVPAT
(Voter Verified Paper Audit Trail) capability.
The optical scan voting systems approaches in this category are
those in which two records are created: A paper and an electronic
record. This system uses Optical Scan Recognition (OCR) to create an
electronic record from the paper record after the paper record has
been directly verified by the voter. The general operation of this
system is:
1. A voter uses a marking device such as a DRE to mark a ballot
and then presses a button to print the marked ballot onto a piece of
paper.
2. The voter directly reviews the paper to ensure its
correctness, and if correct, places the paper record into a scanner
(some procedure would need to be included to handle spoiled
ballots).
3. The scanner converts the paper record into an electronic
format. To reduce errors that may result from scanning the paper
record, the paper records might contain a barcoded representation of
the human readable portion of the ballot.
4. The paper record gets preserved in a ballot box.
No verification of the scanned paper record is performed in the
above approach. One may assume that the scanning process is highly
accurate and can be trusted to create the electronic record
correctly; however it would be preferential for the voter to somehow
verify that the record was, in fact, created correctly.
An electronic voting system with VVPAT (Voter Verified Paper
Audit Trail) capability is similar to that of the optical scan above
but consists typically of a DRE that both creates and records an
electronic record, and a printer that creates a paper audit trail of
the voter's choices. Like the optical scan system, it creates two
distinct representations of the voter's ballot choices: an
electronic record and a paper record.
Typically, a voter would use the voting system (called a DRE-
VVPAT) as follows:
1. A voter makes ballot selections and indicates that his or her
selections are complete.
2. The VVPAT-DRE prints a paper record summary of the voter's
ballot choices. An alternative approach to VVPAT involves printing
the voter's ballot selections as they are made, e.g., a concurrent
or contemporaneous record.
3. The voter inspects and directly verifies that the paper
record matches the displayed electronic record (again, a procedure
would need to be included to handle spoiled ballots).
4. The paper record gets preserved in a ballot box.
Both approaches described here produce paper records that are
verified directly by sight. Voters with sight impairments would
require an accessible device for verification that can produce an
audible representation of the paper record.

1.3 Issues in Handling Multiple Records Produced by Independent
Dual Verification Systems

[[Page 19015]]

The suitability of the records for various types of
auditing, and
How best to address problems if there are errors in the
records.
Whenever an electronic voting system produces multiple records
of votes, there is some possibility that one or more of the records
may not match. Records can be lost, or deliberately or accidentally
damaged, or stolen, or fabricated. Keeping the two records in
correspondence with each other can be made more or less difficult
depending on the technologies used for the records and the
procedures used to handle the records.
As a consequence, it is important to structure the records so
that errors and other anomalies can be readily detected during
audits. There are a number of techniques that can be used, such as
the following:
Associating unique identifiers with corresponding
records, e.g., an individual paper record sharing a unique
identifier with its corresponding electronic record,
Including an identification of the specific voting
system that produced the records, such as a serial number identifier
or by having the voting system digitally sign the records using
public key cryptography,
Including other information about the election and the
precinct or location where the records were created,
Creating checksums of the electronic records and having
the voting system digitally sign the entire sets of records so that
missing or inserted records can be detected, and
Structuring the records in open, publicly documented
formats that can be readily analyzed on different computing
platforms
The ease or relative difficulty with which some types of records
must be handled is also a determining factor in the practical
capability to conduct precise audits, given that some types of
records are better suited to different types of auditing and
different voting environments than others. The factors that make
certain types of records more suitable than others could vary
greatly depending upon many other criteria, both objective and
subjective. For example, paper records may require manual handling
by voters or poll workers and thus be more susceptible to damage or
loss. At the same time, the extent to which the paper records must
be handled will vary depending on the type of voting system in use.
Electronic records may by their nature be more suitable for
automated audits; however electronic records are still subject to
accidental or deliberate damage, loss, and theft.

D.2. Core Characteristics for Independent Verification Systems

This section contains a preliminary set of characteristics for
IDV systems. These characteristics are fundamental in nature and
apply to all categories of IDV systems. They will form the basis for
future requirements for independent verification systems.

2.1 An Independent Dual Verification Voting System Produces Two
Distinct Sets of Records of Ballot Choices Via Interactions With
the Voter Such That One Set of Records Can Be Compared Against the
Other To Check Their Equality of Content
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is the fundamental core definition for IDV
systems. The records can be checked against one another to determine
whether or not the voter's choices were correctly recorded.

2.1.1 The Voter Verifies the Content of Each Record and Either (a)
Verifies at Least One of the Records Directly or (b) Verifies Both
Records Indirectly if the Records Are Each Under the Control of
Independent Processes
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.1.2 The Creation, Storage, and Handling of the Records Are
Sufficiently Separate Such That the Failure or Compromise of One Record
Does Not Cause the Failure or Compromise of Another
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.1.2.1 At Least One Record Is Highly Resistant to Damage or Alteration
and Should Be Capable of Long-Term Storage
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: At least one of the records should be difficult to
alter or damage so that it could be used in case the counted records
are damaged or lost.

2.1.3 The Processes of Verification for the Multiple Records Do Not All
Depend for Their Integrity on the Same Device, Software Module, or
System, and Are Sufficiently Separate Such That Each Record Provides
Evidence of the Voter's Choices Independently of Its Other
Corresponding Record
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.1.4 The Records Can Be Used in Checks of One Another, Such That if
One Set of Records Can Be Used in an Efficient Counting Process, the
Other Set of Records Can Be Used for Checking Its Agreement With the
First Set of Records
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.1.5 The Records Within a Set Are Linked to Their Corresponding
Records in the Other Set By Including a Unique Identifier Within Each
Record That Can Be Used to Identify the Record's Corresponding Record
in the Other Set
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The identifier should serve the purpose of uniquely
identify the record so as to identify duplicates and/or for cross-
checking two record types.

2.1.6 Each Record Includes an Identification of the Voting Site/
Precinct
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If the voting site and precinct are different, both
should be included.

[[Page 19016]]

2.1.7 The Records Include Information Identifying Whether the Balloting
Is Provisional, Early, or on Election Day, and Information That
Identifies the Ballot Style in Use
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

2.1.8 The Records Include a Voting Session Identifier That Is Generated
When the Voting Station Is Placed in Voting Mode and That Can Be Used
To Identify the Records as Being Created During That Voting Session
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If there are several voting sessions on the same
voting station on the same day, the voting session identifiers must
be different. They should be generated from a random number
generator.

2.1.9 The Records Include an Identifier of the Voting System That Is
Unique to That Style of Voting Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The identifier could be a serial number or other
unique ID.

2.1.10 The Cryptographic Software in Independent Verification Voting
Systems Is Approved by the U.S. Government's Cryptographic Module
Validation Program (CMVP) as Applicable
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voting systems may use cryptographic software
for a number of different purposes, including calculating checksums,
encrypting records, authentication, generating random numbers, and
for digital signatures. This software should be reviewed and
approved by the Cryptographic Module Validation Program. There may
be cryptographic voting schemes where the cryptographic algorithms
used are necessarily different from any algorithms that have
approved CMVP implementations, thus CMVP approved software shall be
used where feasible. The CMVP Web site is http://csrc.nist.gov/cryptval
.

D.3. Split Process IDV Systems

This section contains characteristics specific to split process
IDV systems. The characteristics build on and are in addition to the
core characteristics for IDV systems. Split process systems consist
of separate vote capture and verification stations, i.e., two
physical devices. A voter inserts an object called a token into the
capture station to make ballot selections and then takes the token
object to the verification station to review and store his or her
votes. Two records of the vote are created: one on the token object
and one by the verification station.

3.1 Capture and Verification Stations

3.1.1 The Verification Station Is Able To Add Information to the Token
Object But Cannot Change Prior Recorded Information
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.1.2 The Capture and Verification Stations Do Not Permit any
Communications Between Them Except Via the Token Object
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.1.3 The Verification Station Log All Rejected Votes, Including the
Precise Contents of the Votes and the Identifier of the Token Object
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voter could reject and essentially spoil his or
her ballot. This is to prevent the verification station from
recording ballot choices that are different from what was entered at
the capture station.

3.1.4 The Capture and Verification Stations Could Be Purchased From
Different Manufacturers and Could Use Different Operating Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The greater the diversity between the systems, the
less likely they could be compromised by the same threats, e.g.,
software viruses, or by a single conspiracy.

3.2 Data Formats for Token Objects

3.2.1 The Format for Data Written to the Token Object Is Specified and
Publicly Available for Use Without Licensing Fees
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.2.2 The Verification Station Verifies the Correctness of the Data on
the Token Object and Provides an Indication of any Errors to the Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The verification station needs to verify, in
essence, that the data written to the token object was formatted
properly according to the rules of the format's specification and
reject ill-formatted data. It also checks that the votes are
consistent with the voting instructions, e.g., ``vote for one, vote
for two.''

3.2.3 The Record on the Token Object Is Digitally Signed Using a
Private Key Known Only to the Vote Capture Station and Whose Public Key
Is Distributed in an Authenticated Way To Auditing Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.2.4 The Record Created by the Verification Station Is Digitally
Signed Using a Private Key Known Only to the Verification Station and
Whose Public Key Is Distributed in an Authenticated Way To Auditing
Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.2.5 The Capture Station Associates With Each Record of Voter Choices
a Unique Identifier That Is Capable of Being Used To Identify the
Record Uniquely and To Identify Its Corresponding Record Created by the
Verification Station
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The identifier serves the purpose of uniquely
identifying the record to identify duplicates and/or for cross-
checking two record types.

3.2.6 The Records From the Verification Station Are Randomly Shuffled
in Memory and When Exported, so That the Order of the Records Cannot Be
Used To Identify Any Voter
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.2.7 Rejected Token Objects Are Stored Separately From Accepted Memory
Devices for Later Auditing
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.3 Storage and Communications of Records

3.3.1 The Verification Station Exports Its Records of Voter Choices
Accompanied by a Digital Signature on the Entire Set of Electronic
Records and Their Associated Digital Signatures
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[[Page 19017]]

Discussion: This is necessary to determine if records are
missing or substituted.

3.3.2 The Token Objects Are Carried in a Physically Secure Way, Using
Chain-of-Custody Mechanisms To Ensure Their Integrity
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

3.3.3 The Records From Each Station Are Randomly Shuffled, so That an
Attacker Learning the Contents of Those Records at Any Point in the
Voting Process Can Learn Nothing About the Order of Votes Cast
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

D.4. Witness IDV Systems

This section contains preliminary characteristics for Witness
IDV systems. They are consistent with the definition of IDV systems
from Section 6.0 and build on the core characteristics for IDV
systems.
Witness IDV systems are composed of two physically separate
devices: the vote capture station that captures and stores records
of voters' choices, and the witness device that captures voter
verifications of the records at the vote capture station. Because
there are two devices, a number of the definitions for split
verification systems apply equally well to witness systems. Because
the vote capture station is in essence a DRE (with or without VVPAT
capability), a number of the definitions for VVPAT that are specific
to DRE systems also apply to vote capture stations. A witness system
fits somewhat loosely in the independent verification category
because the voter performs only an indirect verification of ballot
choices at the DRE. It is important that the witness device be
tested extensively for accuracy and reliability and that
malfunctions in the device be made immediately obvious to voters and
poll workers.

4.1 A Witness Device Records Only a Voter's Verification at a
Voting Station and Stores the Record so That It Can Be Used for
Audit and Recounts as Applicable
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

4.2 A Witness Device Acts as a Passive Device That Cannot Perform
any Operation With Respect to the Voting Station Other Than To
Capture the Voter's Ballot Choices as the Voter Verifies Them
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The witness device is synchronized with the voter
verification of the ballot choices.

4.3 A Witness Device, if Attached to the Voting Station, Is
Attached Such That it Can Capture Only the Voter's Verification of
Ballot Choices
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, the witness device could be connected
only to the display unit and not the vote capture station's memory
or disk drive.

4.4 The Voting Station Is Not Able To Detect in Its Function
Whether a Witness Device Is Electrically Connected or in Operation
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: If the witness device is connected to or attached
electrically to the vote capture station, the capture station is not
able to determine or be aware in its function that a witness device
is attached.

4.5 The Witness Device Operates Properly With Most if Not All
Electronic Voting Systems Functioning as Voting Stations
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is desirable but may require some degree of
openness in witness device specifications to enable the desired
compatibility.

4.6 The Witness Device Is Not Designed or Built or Manufactured by
the Same Manufacturer of the Voting Station to Which it Is Attached
[GRAPHIC] [TIFF OMITTED] TN12AP06.010

4.7 Because Voters Must Trust That the Witness Device Records Their
Verifications Accurately, Assessments of Its Software and
Functionality Are Straightforward, Readily Performed, and Include
Extensive Evaluation and Penetration Testing Above and Beyond What
May Be Performed on Voting Systems That Do Not Contain Witness
Devices
[GRAPHIC] [TIFF OMITTED] TN12AP06.010

Discussion: Witness device manufacturers will need to document
their systems extensively and subject them to highly stringent
testing.

4.8 Because Voters Must Trust That the Witness Device Records Their
Verifications Accurately, the Results of Witness System Assessments
Are Made Publicly Available
[GRAPHIC] [TIFF OMITTED] TN12AP06.010

4.9 A Voter Should Be Able To Inspect the Record of the Voter's
Verification Upon the Voter's Request
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: It is desirable that a voter have some capability to
verify that the witness device is operating as specified.

4.10 The Witness Device Clearly Indicates Any Malfunction in a Way
That Is Obvious to Poll Workers and Voters
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This serves to ensure that voting cannot continue if
the witness device is not operating or is malfunctioning.

4.11 The Records Captured by the Witness Device Are Able To Be Used
in Highly Accurate Verifications of the Voting Records of the
Voting Station
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

4.12 The Records Contain Unique Identifiers That Correspond to
Records Stored by the Voting Station
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

4.13 The Records Are Digitally Signed by the Witness Device so That
the Integrity and Authenticity of Its Records Can Be Verified
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

4.14 A Witness Device Is Able To Export Its Records in an Open,
Nonproprietary Format Such That the Records Can Be Used in
Automated Audits
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

[[Page 19018]]

4.15 The Records Are Stored in the Witness Device and Exported Such
That Voter Privacy Is Protected, e.g., by Making the Order of the
Records Randomly Determined
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

D.5. End to End (Cryptographic) IDV Systems

This section contains very preliminary definitions for End to
End (or cryptographic-based) IDV systems. They are consistent with
the characteristics of IDV systems and build on the core
characteristics of IDV systems.
End to end voting systems use cryptographic mechanisms as a
substitute for some of the physical, computer-security, or
procedural mechanisms used to secure other voting systems. Some
auditing procedures normally performed by voting officials at the
tabulation center can be done by voters or their designated
representatives, using receipts issued by the voting system that
work in conjunction with the cryptographic mechanisms. Typically,
multiple individuals, known as designated trustees, hold key
information that is combined to form encryption and decryption keys;
thus, no one person is able to encrypt or decrypt. Several types of
cryptographic voting approaches have been proposed or implemented,
with varying properties. There are many cryptographic techniques
(such as secure multiparty computation and homomorphic) that could
be applied in novel ways in future voting systems.
End to end systems use cryptographic mechanisms as a substitute
for some of the physical, computer security, and procedural
mechanisms used to secure voting systems. These cryptographic
mechanisms can be used by a voter to verify that ballot choices were
recorded correctly and counted in the election.

5.1 End to End Systems Use Cryptographic Mechanisms as a Substitute
for Some of the Physical, Computer Security, and Procedural
Mechanisms Used To Secure Voting Systems. These Cryptographic
Mechanisms Can Be Used by a Voter To Verify That Ballot Choices
Were Recorded Correctly and Counted in the Election
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: There are potentially many types of end to end
systems that could perform a variety of different functions.

5.2 End to End Systems Record Voters Ballot Choices at an
Electronic Voting System and Encrypt the Records of Votes for Later
Counting by Designated Trustees
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The voting station would operate much as a DRE.

5.3 End to End Systems Produce a Receipt That Can Be Used by the
Voter in Some Process Made Available by Voting Officials That Would
Enable the Voter to Verify That the Voter's Ballot Choices Were
Recorded Correctly and Counted in the Election
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: The receipt could have a variety of different forms
but likely would be printed on paper for the voter's ease of
handling.

5.4 No One Designated Trustee Is Able to Decrypt the Records;
Decryption of the Records Is Performed by a Process That Involves
Multiple Designated Trustees
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: For example, multiple keys could be combined to
decrypt the records.

5.5 The Receipt Preserves Voter Privacy by Not Containing any
Information That Can Be Used To Show the Voter's Choices
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.6 The Process Used To Verify That Ballot Choices Were Recorded
Correctly or Counted in the Election Preserves Voter Privacy by not
Revealing any Information That Can Be Used to Show the Voter's
Choices
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.7 End to End Systems Store Backup Records of Voter's Ballot
Choices That Can Be Used in Contingencies Such as Damage to or Loss
of Its Counted Records
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: This is necessary because the handling of the
encrypted records requires the same chain of custody procedures as
records produced by other voting systems and are thus subject to
loss or damage. This could be paper for example.

5.8 The Backup Records Contain Unique Identifiers That Correspond
to Unique Identifiers in Its Counted Records, and the Backup
Records Are Digitally Signed so That They Can Be Verified for Their
Authenticity and Integrity In Audits
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.9 Cryptographic Software in End to End Systems Is Documented
Thoroughly and Subject To Extensive Verification Testing for
Correctness. The Documentation Includes Extensive Discussion of How
Cryptographic Keys Are To Be Generated, Distributed, Managed, Used,
Certified, and Destroyed
[GRAPHIC] [TIFF OMITTED] TN12AP06.010

Discussion: The correctness of the system depends on the
correctness of the cryptographic algorithms and their
implementations. Thus, rigorous testing is necessary.

5.10 Vote Capture Stations Used in End to End Systems Meet All
Security, Usability, and Accessibility Requirements for Similar
Stations in Other Voting Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.11 Reliability, Usability, and Accessibility Requirements for
Printers in Other Voting Systems Apply As Well to Receipt Printers
Used in End to End Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

5.12 Trustee Systems Are Subject to the Same Evaluations and
Assessments as Other Voting Systems
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Trustee systems include systems to perform
cryptographic functions such as encrypting or decrypting votes.

5.13 Systems for Verifying That Voters' Ballots Were Recorded
Properly and Counted in the Election Are Implemented in a Robust
Secure Manner
[GRAPHIC] [TIFF OMITTED] TN12AP06.003

Discussion: Many of the cryptographic approaches have a ``public
append-only bulletin board'' as a component; this is an important
part of the system and needs to be implemented in a robust secure
manner.

Volume II, Section 1

Table of Contents

1 Introduction

1.1 Objectives and Usage of Volume II of the Voting Systems
Standards
1.2 General Contents of Volume II
1.3 Qualification Testing Focus
1.4 Qualification Testing Sequence
1.5 Evolution of Testing

[[Page 19019]]

1.6 Outline of Contents

1 Introduction

1.1 Objectives and Usage of Volume II of the Voting Systems
Standards

Volume II, Voting System Qualification Testing Standards, is a
complementary document to Volume I, Voting System Performance
Standards. While Section 9 of Volume I provides an overview of the
qualification testing process performed by the Independent Test
Authorities (ITAs), Volume II provides specific detail about the
process that is necessary for ITAs, vendors, and election officials
participating in the qualification process. The Standards envision a
diverse set of users for Volume II, including:
Vendors: Voting system vendors will use Volume II to
guide the design, construction, documentation, internal testing, and
maintenance of voting systems to ensure conformance with the
Standards. Vendors will also use Volume II to help define the
obligations of organizations that support the vendor's system, such
as suppliers, testers, and consultants.
Independent Testing Authorities: Testing authorities
certified to qualify systems will use Volume II to guide the testing
of voting systems and preparation of test reports. Laboratories and
other parties interested in becoming ITAs can use Volume II to
understand the requirements and obligations placed on the ITAs
involved in the process.
Election officials: Voting officials in many
jurisdictions will use Volume II to guide system certification,
procurement and acceptance requirements and processes, which may
include additional requirements and adjustments to those
requirements included in the Standards.

1.2 General Contents of Volume II

To support these primary users of the Standards, Volume II
provides:
a. A discussion of the general sequencing of tests performed by
the ITAs: Volume II identifies the tests where sequencing is
important and provides such required sequences. Volume II also
indicates other tests that may be conducted in parallel.
b. A detailed description of the information required to be
submitted by voting system vendors in the Technical Data Package
(TDP): The TDP is a comprehensive set of documents that describe
system design specifications, operating procedures, system testing
information, facility and resource requirements for system
operations, system maintenance instructions for jurisdictions, and
vendor practices for quality assurance and configuration management
that underlie the development and update of the system. The TDP
focuses predominantly on the required documentation contents,
providing flexibility to vendors to determine the best format for
meeting the content requirements.
c. Delineation of specific system tests to be conducted by the
ITAs: Volume II identifies specific tests that are to be conducted
relating to system components and to the integrated system as a
whole. Tests are defined for system functionality, hardware,
software, telecommunications, and security that address the
performance standards delineated in Volume I.
d. Delineation of specific examinations of other information
provided by the vendor: Volume II identifies the criteria to be used
by the ITAs in conducting examinations of the information submitted
in the TDP. These criteria address the documentation provided in the
TDP, including documentation of the system and related operational
procedures as well as vendor practices for quality assurance and
configuration management.
e. Description of process for handling failures: A system may
fail to pass one or more of the tests and examinations performed by
the ITAs. Volume II describes the practices to be used by the ITAs
when the system or its documentation fails a test or examination,
including the nature and depth of re-testing required for
corrections submitted by the vendor.
f. Outline of Qualification Test Report. Volume II provides an
outline of the report issued by the ITAs at the conclusion of
testing, providing the specific requirements for this report.

1.3 Qualification Testing Focus

Qualification tests focus on multiple aspects of the voting
system and the process for development and maintenance. Although
multiple ITAs may conduct qualification testing, with each ITA
conducting tests in its areas of expertise, the focus of their
combined activities remains the same. Overall, qualification testing
focuses on:
a. The functional capabilities of the system to support specific
election activities performed by system users, including election
officials and voters, as defined in Volume I, Section 2 of the
Standards;
b. The performance capabilities of the system that ensure
accuracy, integrity, and reliability of system operations and the
election activities that rely on them, as defined in Volume I,
Sections 3, 4, 5 and 6 of the Standards;
c. The system development and maintenance processes and related
quality assurance activities performed by the vendor to ensure
system quality, as addressed in Volume I, Section 7 of the
Standards;
d. The configuration management activities used to control the
development and modification of the system and its individual
components, and maintain accurate information about the version and
status of the system and its components throughout the system life
cycle, as addressed in Volume I, Section 8 of the Standards; and
e. The documentation developed and maintained by the vendor to
support system development, testing, installation, maintenance and
operation, as addressed by the TDP described in Volume II, Section
2.

1.4 Qualification Testing Sequence

The overall qualification test process progresses through
several stages involving pre-testing, testing, and post-testing
activities as described in Volume I, Section 9 of the Standards.
Whereas Volume I describes the flow of the overall process, Volume
II focuses on the details of activities conducted by the ITA and
activities conducted by the vendor to facilitate testing and respond
to errors, anomalies, and other findings of concern during the test
process.
Qualification testing involves a series of physical tests and
other examinations that are conducted in a particular sequence. This
sequence is intended to maximize overall testing effectiveness, as
well as conduct testing in as efficient a manner as possible. The
ITA follows the general sequence of activities indicated below. Note
that test errors and anomalies are communicated to the vendor
throughout the process.
a. Initial examination of the system and TDP provided by the
vendor to ensure that all components and documentation needed to
conduct testing have been submitted, and to help determine the scope
and level of effort of testing needed;
b. Development of a detailed system test plan that reflects the
scope and complexity of the system, and the status of system
qualification (i.e., initial qualification or re-qualification);
c. Operational testing of hardware components, including
environmental tests, to ensure that operational performance
requirements are achieved;
d. Functional and performance testing of hardware components;
e. Examination of the vendor's Quality Assurance Program and
Configuration Management Plan;
f. Code review for selected software components;
g. Functional and performance testing of software components;
h. System installation testing and testing of related
documentation for system installation and diagnostic testing;
i. Functional and performance testing of the integrated system,
including testing of the full scope of system functionality,
performance tests for telecommunications and security; and
examination and testing of the System Operations Manual;
j. Examination of the System Maintenance Manual;
k. Witnessing of a system `build' conducted by the vendor to
conclusively establish the system version and components being
tested; and
l. Preparation of the Qualification Test Report.

1.5 Evolution of Testing

The ITA will conduct extensive tests on a voting system to
evaluate it against the requirements of the Standards. Taking
advantage of the experience gained in examining other voting
systems, ITAs will design tests specifically for the system design,
configuration, and documentation provided by the vendor.
Additionally, new threats may be identified that are not directly
addressed by the Standards or the system. As new threats to a voting
system are discovered, either during the system's operation or
during the operation of other computer-based systems that use
technologies comparable to those of another voting system, ITAs
shall expand the tests used for system security to address the
threats that are applicable to a particular design of voting system.

1.6 Outline of Contents

Volume II of the Voting Systems Standards is organized as
follows:

[[Page 19020]]

Section 2 describes the requirements for the Technical
Data Package;
Section 3 describes functionality testing;
Sections 4 and 5 describe specific testing standards
for hardware and software;
Section 6 describes standards for testing the fully
integrated system, including telecommunications and security
capabilities, and the documentation used to operate the system;
Section 7 describes the standards for examining the
documentation of vendor practices for quality assurance and
configuration management;
Appendix A provides an outline for the Qualification
Test Plan;
Appendix B provides an outline for the Qualification
Test Report; and
Appendix C describes the guiding principles used to
design the voting system qualification testing process performed by
ITAs.

Volume II, Section 2

Table of Contents

2 Technical Data Package

2.1 Scope
2.1.1 Content and Format
2.1.1.1 Required Content for Initial Qualification
2.1.1.2 Required Content for System Changes and Re-qualification
2.1.1.3 Format
2.1.2 Other Uses for Documentation
2.1.3 Protection of Proprietary Information
2.2 System Overview
2.2.1 System Description
2.2.2 System Performance
2.3 System Functionality Description
2.4 System Hardware Specification
2.4.1 System Hardware Characteristics
2.4.2 Design and Construction
2.5 Software Design and Specification
2.5.1 Purpose and Scope
2.5.2 Applicable Documents
2.5.3 Software Overview
2.5.4 Software Standards and Conventions
2.5.5 Software Operating Environment
2.5.5.1 Hardware Environment and Constraints
2.5.5.2 Software Environment
2.5.6 Software Functional Specification
2.5.6.1 Configurations and Operating Modes
2.5.6.2 Software Functions
2.5.7 Programming Specifications
2.5.7.1 Programming Specifications Overview
2.5.7.2 Programming Specifications Details
2.5.8 System Database
2.5.9 Interfaces
2.5.9.1 Interface Identification
2.5.9.2 Interface Description
2.5.10 Appendices
2.6 System Security Specification
2.6.1 Access Control Policy
2.6.2 Access Control Measures
2.6.3 Equipment and Data Security
2.6.4 Software Installation
2.6.5 Telecommunications and Data Transmission Security
2.6.6 Other Elements of an Effective Security Program
2.7 System Test and Verification Specification
2.7.1 Development Test Specifications
2.7.2 Qualification Test Specifications
2.8 System Operations Procedures
2.8.1 Introduction
2.8.2 Operational Environment
2.8.3 System Installation and Test Specification
2.8.4 Operational Features
2.8.5 Operating Procedures
2.8.6 Operations Support
2.8.7 Appendices
2.9 System Maintenance Procedures
2.9.1 Introduction
2.9.2 Maintenance Procedures
2.9.2.1 Preventive Maintenance Procedures
2.9.2.2 Corrective Maintenance Procedures
2.9.3 Maintenance Equipment
2.9.4 Parts and Materials
2.9.4.1 Common Standards
2.9.4.2 Paper-Based Systems
2.9.5 Maintenance Facilities and Support
2.9.6 Appendices
2.10 Personnel Deployment and Training Requirements
2.10.1 Personnel
2.10.2 Training
2.11 Configuration Management Plan
2.11.1 Configuration Management Policy
2.11.2 Configuration Identification
2.11.3 Baseline, Promotion, and Demotion Procedures
2.11.4 Configuration Control Procedures
2.11.5 Release Process
2.11.6 Configuration Audits
2.11.7 Configuration Management Resources
2.12 Quality Assurance Program
2.12.1 Quality Assurance Policy
2.12.2 Parts & Materials Special Tests and Examinations
2.12.3 Quality Conformance Inspections
2.12.4 Documentation
2.13 System Change Notes

2 Technical Data Package

2.1 Scope

This section contains a description of vendor documentation
relating to the voting system that shall be submitted with the system
as a precondition of qualification testing. These items are necessary
to define the product and its method of operation; to provide technical
and test data supporting the vendor's claims of the system's functional
capabilities and performance levels; and to document instructions and
procedures governing system operation and field maintenance. Other
items relevant to the system evaluation shall be submitted along with
this documentation (such as disks, tapes, source code, object code, and
sample output report formats).
Both formal documentation and notes of the vendor's system
development process shall be submitted for qualification tests.
Documentation outlining system development permits assessment of the
vendor's systematic efforts to test the system and correct defects.
Inspection of this process also enables the design of a more precise
qualification test plan. If the vendor's developmental test data is
incomplete, the test agency shall design and conduct the appropriate
tests.
2.1.1 Content and Format
The content of the Technical Data Package (TDP) is intended to
collect clear, complete descriptions of the following information about
the system:
Overall system design, including subsystems, modules and
the interfaces among them;
Specific functional capabilities provided by the system;
Performance and design specifications;
Design constraints, applicable standards, and
compatibility requirements;
Personnel, equipment, and facility requirements for system
operation, maintenance, and logistical support;
Vendor practices for assuring system quality during the
system's development and subsequent maintenance; and
Vendor practices for managing the configuration of the
system during development and for modifications to the system
throughout its life cycle.
The vendor shall list all documents controlling the design,
construction, operation, and maintenance of the system. Documents shall
be listed in order of precedence.
2.1.1.1 Required Content for Initial Qualification
At minimum, the TDP shall contain the following documentation:
a. System configuration overview;
b. System functionality description;
c. System hardware specifications;
d. Software design and specifications;
e. System test and verification specifications;
f. System security specifications;
g. User/system operations procedures;
h. System maintenance procedures;
i. Personnel deployment and training requirements;
j. Configuration management plan;
k. Quality assurance program; and
l. System change notes.
2.1.1.2 Required Content for System Changes and Re-qualification
For systems seeking re-qualification, vendors shall submit System
Change Notes as described in section 2.13, as well as current versions
of all documents that have been updated to reflect system changes.
Systems in existence at the time the revised standards are released
may not

[[Page 19021]]

have all required developmental documentation. When such a system is
subject to evaluation as a result of system modification, the vendor
shall provide what information they can.
Vendors may also submit other information relevant to the
evaluation of the system, such as documentation of tests performed by
other independent test authorities and records of the system's
performance history, if any.
2.1.1.3 Format
The requirements for formatting the TDP are general in nature;
specific format details are of the vendor's choosing. Other items
submitted by the vendor, such as documentation of tests conducted by
other test authorities, performance history, failure analysis, and
corrective action may be provided in a format of the vendor's choosing.
The TDP shall include a detailed table of contents for the required
documents, an abstract of each document and a listing of each of the
informational sections and appendices presented. A cross-index shall be
provided indicating the portions of the documents that are responsive
to documentation requirements for any item presented using the vendor's
format.
2.1.2 Other Uses for Documentation
Although all of the TDP documentation is required for qualification
testing, some of these same items may also be required during the state
certification process and local level acceptance testing. Therefore, it
is recommended that the technical documentation required for
certification and acceptance testing be deposited in escrow.
2.1.3 Protection of Proprietary Information
The vendor shall identify all documents, or portions of documents,
containing proprietary information not approved for public release. Any
person or test agency receiving proprietary information shall agree to
use it solely for the purpose of analyzing and testing the system, and
shall agree to refrain from otherwise using the proprietary information
or disclosing it to any other person or agency without the prior
written consent of the vendor, unless disclosure is legally compelled.

2.2 System Overview

In the system overview, the vendor shall provide information that
enables the test authority to identify the functional and physical
components of the system, how the components are structured, and the
interfaces between them.
2.2.1 System Description
The system description shall include written descriptions, drawings
and diagrams that present:
a. A description of the functional components (or subsystems) as
defined by the vendor (e.g., environment, election management and
control, vote recording, vote conversion, reporting, and their
interconnection);
b. A description of the operational environment of the system that
provides an overview of the hardware, software, and communications
structure;
c. A theory of operation that explains each system function, and
how the function is achieved in the design;
d. Descriptions of the functional and physical interfaces between
subsystems and components;
e. Identification of all COTS hardware and software products and
communications services used in the development and/or operation of the
voting system, identifying the name, vendor and version used for each
such component, including:
(1) Operating systems;
(2) Database software;
(3) Communications routers;
(4) Modem drivers; and
(5) Dial-up networking software;
f. Interfaces among internal components, and interfaces with
external systems. For components that interface with other components
for which multiple products may be used, the TDP shall provide an
identification of:
(1) File specifications, data objects, or other means used for
information exchange; and
(2) The public standard used for such file specifications, data
objects, or other means; and
g. Benchmark directory listings for all software (including
firmware elements) and associated documentation included in the
vendor's release in order of how each piece of software would normally
be installed upon setup and installation.
2.2.2 System Performance
The vendor shall provide system performance information that
includes descriptions of:
a. The performance characteristics of each operating mode and
function in terms of expected and maximum speed, throughput capacity,
maximum volume (maximum number of voting positions and maximum number
of ballot styles supported), and processing frequency;
b. Quality attributes such as reliability, maintainability,
availability, usability, and portability;
c. Provisions for safety, security, privacy, and continuity of
operation; and
d. Design constraints, applicable standards, and compatibility
requirements.

2.3 System Functionality Description

The vendor shall declare the scope of the system's functional
capabilities, thereby establishing the performance, design, test,
manufacture, and acceptance context for the system.
The vendor shall provide a listing of the system's functional
processing capabilities, encompassing capabilities required by the
Standards and any additional capabilities provided by the system. This
listing shall provide a simple description of each capability. Detailed
specifications shall be provided in other documentation required for
the TDP as indicated by the standards for that documentation.
a. The vendor shall organize the presentation of required
capabilities in a manner that corresponds to the structure and sequence
of functional capabilities indicated in Volume I, Section 2 of the
Standards. The contents of Volume I Section 2 may be used as the basis
for a checklist whereby the vendor indicates the specific functions
provided and those not provided by the system;
b. Additional capabilities shall be clearly indicated. They may be
presented using the same structure as that used for required
capabilities (i.e., overall system capabilities, pre-voting functions,
voting functions, post-voting functions), or may be presented in
another format of the vendor's choosing;
c. Required capabilities that may be bypassed or deactivated during
installation or operation by the user shall be clearly indicated;
d. Additional capabilities that function only when activated during
installation or operation by the user shall be clearly indicated; and
e. Additional capabilities that normally are active but may be
bypassed or deactivated during installation or operation by the user
shall be clearly indicated.

2.4 System Hardware Specification

The vendor shall expand on the system overview by providing
detailed specifications of the hardware components of the system,
including specifications of hardware used to support the
telecommunications capabilities of the system, if applicable.
2.4.1 System Hardware Characteristics
The vendor shall provide a detailed discussion of the
characteristics of the system, indicating how the hardware meets
individual requirements defined

[[Page 19022]]

in Volume I, Sections 3, 4, 5 and 6 of the Standards, including:
a. Performance characteristics: This discussion addresses basic
system performance attributes and operational scenarios that describe
the manner in which system functions are invoked, describe
environmental capabilities, describe life expectancy, and describe any
other essential aspects of system performance;
b. Physical characteristics: This discussion addresses suitability
for intended use, requirements for transportation and storage, health
and safety criteria, security criteria, and vulnerability to adverse
environmental factors;
c. Reliability: This discussion addresses system and component
reliability stated in terms of the systems operating functions, and
identification of items that require special handling or operation to
sustain system reliability;
d. Maintainability: Maintainability represents the ease with which
maintenance actions can be performed based on the design
characteristics of equipment and software and the processes the vendor
and election officials have in place for preventing failures and for
reacting to failures. Maintainability includes the ability of equipment
and software to self-diagnose problems and make non-technical election
workers aware of a problem. Maintainability also addresses a range of
scheduled and unscheduled events; and
e. Environmental conditions: This discussion addresses the ability
of the system to withstand natural environments, and operational
constraints in normal and test environments, including all requirements
and restrictions regarding electrical service, telecommunications
services, environmental protection, and any additional facilities or
resources required to install and operate the system.
2.4.2 Design and Construction
The vendor shall provide sufficient data, or references to data, to
identify unequivocally the details of the system configuration
submitted for qualification testing. The vendor shall provide a list of
materials and components used in the system and a description of their
assembly into major system components and the system as a whole.
Paragraphs and diagrams shall be provided that describe:
a. Materials, processes, and parts used in the system, their
assembly, and the configuration control measures to ensure compliance
with the system specification;
b. The electromagnetic environment generated by the system;
c. Operator and voter safety considerations, and any constraints on
system operations or the use environment;
d. Human engineering considerations, including provisions for
access by disabled voters.

2.5 Software Design and Specification

The vendor shall expand on the system overview by providing
detailed specifications of the software components of the system,
including software used to support the telecommunications capabilities
of the system, if applicable.
2.5.1 Purpose and Scope
The vendor shall describe the function or functions that are
performed by the software programs that comprise the system, including
software used to support the telecommunications capabilities of the
system, if applicable.
2.5.2 Applicable Documents
The vendor shall list all documents controlling the development of
the software and its specifications. Documents shall be listed in order
of precedence.
2.5.3 Software Overview
The vendor shall provide an overview of the software that includes
the following items:
a. A description of the software system concept, including specific
software design objectives, and the logic structure and algorithms used
to accomplish these objectives;
b. The general design, operational considerations, and constraints
influencing the design of the software;
c. Identification of all software items, indicating items that
were:
(1) Written in-house;
(2) Procured and not modified; and
(3) Procured and modified including descriptions of the
modifications to the software and to the default configuration options;
d. Additional information for each item that includes:
(1) Item identification;
(2) General description;
(3) Software requirements performed by the item;
(4) Identification of interfaces with other items that provide data
to, or receive data from, the item; and
(5) Concept of execution for the item;
The vendor shall also include a certification that procured
software items were obtained directly from the manufacturer or a
licensed dealer or distributor.
2.5.4 Software Standards and Conventions
The vendor shall provide information that can be used by an ITA or
state certification board to support software analysis and test design.
The information shall address standards and conventions developed
internally by the vendor as well as published industry standards that
have been applied by the vendor. The vendor shall provide information
that addresses the following standards and conventions:
a. System development methodology;
b. Software design standards, including internal vendor procedures;
c. Software specification standards, including internal vendor
procedures;
d. Software coding standards, including internal vendor procedures;
e. Software testing and verification standards, including internal
vendor procedures, that can assist in determining the program's
correctness and ACCEPT/REJECT criteria; and
f. Quality assurance standards or other documents that can be used
by the ITA to examine and test the software. These documents include
standards for program flow and control charts, program documentation,
test planning, and for test data acquisition and reporting.
2.5.5 Software Operating Environment
This section shall describe or make reference to all operating
environment factors that influence the software design.
2.5.5.1 Hardware Environment and Constraints
The vendor shall identify and describe the hardware characteristics
that influence the design of the software, such as:
a. The logic and arithmetic capability of the processor;
b. Memory read-write characteristics;
c. External memory device characteristics;
d. Peripheral device interface hardware;
e. Data input/output device protocols; and
f. Operator controls, indicators, and displays.
2.5.5.2 Software Environment
The vendor shall identify the compilers or assemblers used in the
generation of executable code, and describe the operating system or
system monitor.
2.5.6 Software Functional Specification
The vendor shall provide a description of the operating modes of

[[Page 19023]]

the system and of software capabilities to perform specific functions.
2.5.6.1 Configurations and Operating Modes
The vendor shall describe all software configurations and operating
modes of the system, such as ballot preparation, election programming,
preparation for opening the polling place, recording votes and/or
counting ballots, closing the polling place, and generating reports.
For each software function or operating mode, the vendor shall provide:
a. A definition of the inputs to the function or mode (with
characteristics, tolerances or acceptable ranges, as applicable);
b. An explanation of how the inputs are processed; and
c. A definition of the outputs produced (again, with
characteristics, tolerances, or acceptable ranges as applicable).
2.5.6.2 Software Functions
The vendor shall describe the software's capabilities or methods
for detecting or handling:
a. Exception conditions;
b. System failures;
c. Data input/output errors;
d. Error logging for audit record generation;
e. Production of statistical ballot data;
f. Data quality assessment; and
g. Security monitoring and control.
2.5.7 Programming Specifications
The vendor shall provide in this section an overview of the
software design, its structure, and implementation algorithms and
detailed specifications for individual software modules.
2.5.7.1 Programming Specifications Overview
This overview shall include such items as flowcharts, HIPOs, data
flow diagrams, and other graphical techniques that facilitate
understanding of the programming specifications. This section shall be
prepared to facilitate understanding of the internal functioning of the
individual software modules. Implementation of the functions shall be
described in terms of the software architecture, algorithms, and data
structures.
2.5.7.2 Programming Specifications Details
The programming specifications shall describe individual software
modules and their component units, if applicable. For each module and
unit, the vendor shall provide the following information:
a. Module and unit design decisions, if any, such as algorithms
used;
b. Any constraints, limitations, or unusual features in the design
of the software module or unit;
c. The programming language to be used and rationale for its use if
other than the specified module or unit language;
d. If the software module or unit consists of or contains
procedural commands (such as menu selections in a database management
system (DBMS) for defining forms and reports, on-line DBMS queries for
database access and manipulation, input to a graphical user interface
(GUI) builder for automated code generation, commands to the operating
system, or shell scripts), a list of the procedural commands and
reference to user manuals or other documents that explain them;
e. If the software module or unit contains, receives, or outputs
data, a description of its inputs, outputs, and other data elements as
applicable. (Section 2.5.9 describes the requirements for documenting
system interfaces.) Data local to the software module or unit shall be
described separately from data input to or output from the software
module or unit;
f. If the software module or unit contains logic, the logic to be
used by the software unit, including, as applicable:
(1) Conditions in effect within the software module or unit when
its execution is initiated;
(2) Conditions under which control is passed to other software
modules or units;
(3) Response and response time to each input, including data
conversion, renaming, and data transfer operations;
(4) Sequence of operations and dynamically controlled sequencing
during the software module's or unit's operation, including:
(i) The method for sequence control;
(ii) The logic and input conditions of that method, such as timing
variations, priority assignments;
(iii) Data transfer in and out of memory; and
(iv) The sensing of discrete input signals, and timing
relationships between interrupt operations within the software module
or unit; and
(5) Exception and error handling; and
g. If the software module is a database, provide the information
described in Volume II, Section 2.5.8.
2.5.8 System Database
The vendor shall identify and provide a diagram and narrative
description of the system's databases, and any external files used for
data input or output. The information provided shall include for each
database or external file:
a. The number of levels of design and the names of those levels
(such as conceptual, internal, logical, and physical);
b. Design conventions and standards (which may be incorporated by
references) needed to understand the design;
c. Identification and description of all database entities and how
they are implemented physically (e.g., tables, files, etc.);
d. Entity relationship diagram and description of relationships;
and
e. Details of table, record or file contents (as applicable) to
include individual data elements and their specifications, including:
(1) Names/identifiers;
(2) Data type (alphanumeric, integer, etc.);
(3) Size and format (such as length and punctuation of a character
string);
(4) Units of measurement (such as meters, dollars, nanoseconds);
(5) Range or enumeration of possible values (such as 0-99);
(6) Accuracy (how correct) and precision (number of significant
digits);
(7) Priority, timing, frequency, volume, sequencing, and other
constraints, such as whether the data element may be updated and
whether business rules apply;
(8) Security and privacy constraints; and
(9) Sources (setting/sending entities) and recipients (using/
receiving entities); and
f. For external files, a description of the procedures for file
maintenance, management of access privileges, and security.
2.5.9 Interfaces
The vendor shall identify and provide a complete description of all
internal and external interfaces, using a combination of text and
diagrams.
2.5.9.1 Interface Identification
For each interface identified in the system overview, the vendor
shall:
a. Provide a unique identifier assigned to the interface;
b. Identify the interfacing entities (systems, configuration items,
users, etc.) by name, number, version, and documentation references, as
applicable; and
c. Identify which entities have fixed interface characteristics
(and therefore impose interface requirements on interfacing entities)
and which are being developed or modified (thus having interface
requirements imposed on them).

[[Page 19024]]

2.5.9.2 Interface Description
For each interface identified in the system overview, the vendor
shall provide information that describes:
a. The type of interface (such as real-time data transfer, storage-
and-retrieval of data, etc.) to be implemented;
b. Characteristics of individual data elements that the interfacing
entity(ies) will provide, store, send, access, receive, etc., such as:
(1) Names/identifiers;
(2) Data type (alphanumeric, integer, etc.);
(3) Size and format (such as length and punctuation of a character
string);
(4) Units of measurement (such as meters, dollars, nanoseconds);
(5) Range or enumeration of possible values (such as 0-99);
(6) Accuracy (how correct) and precision (number of significant
digits);
(7) Priority, timing, frequency, volume, sequencing, and other
constraints, such as whether the data element may be updated and
whether business rules apply;
(8) Security and privacy constraints; and
(9) Sources (setting/sending entities) and recipients (using/
receiving entities);
c. Characteristics of communication methods that the interfacing
entity(ies) will use for the interface, such as:
(1) Communication links/bands/frequencies/media and their
characteristics;
(2) Message formatting;
(3) Flow control (such as sequence numbering and buffer
allocation);
(4) Data transfer rate, whether periodic/aperiodic, and interval
between transfers;
(5) Routing, addressing, and naming conventions;
(6) Transmission services, including priority and grade; and
(7) Safety/security/privacy considerations, such as encryption,
user authentication, compartmentalization, and auditing;
d. Characteristics of protocols the interfacing entity(ies) will
use for the interface, such as:
(1) Priority/layer of the protocol;
(2) Packeting, including fragmentation and reassembly, routing, and
addressing;
(3) Packeting, including fragmentation and reassembly, routing, and
addressing;
(4) Legality checks, error control, and recovery procedures;
(5) Synchronization, including connection establishment,
maintenance, termination; and
(6) Status, identification, and any other reporting features; and
e. Other characteristics, such as physical compatibility of the
interfacing entity(ies) (dimensions, tolerances, loads, voltages, plug
compatibility, etc.).
2.5.10 Appendices
The vendor may provide descriptive material and data supplementing
the various sections of the body of the Software Specifications. The
content and arrangement of appendices shall be at the discretion of the
vendor. Topics recommended for amplification or treatment in appendix
form include:
a. Glossary: A listing and brief definition of all software module
names and variable names, with reference to their locations in the
software structure. Abbreviations, acronyms, and terms should be
included, if they are either uncommon in data processing and software
development or are used in an unorthodox semantic;
b. References: A list of references to all related vendor
documents, data, standards, and technical sources used in software
development and testing; and
c. Program Analysis: The results of software configuration analysis
algorithm analysis and selection, timing studies, and hardware
interface studies that are reflected in the final software design and
coding.

2.6 System Security Specification

Vendors shall submit a system security specification that addresses
the security requirements of Volume I, Section 6 of the Standards. This
specification shall describe the level of security provided by the
system in terms of the specific security risks addressed by the system,
the means by which each risk is addressed, the process used to test and
verify the effective operation of security capabilities and, for
systems that use public telecommunications networks as defined in
Volume I, Section 5, the means used to keep the security capabilities
of the system current to respond to the evolving threats against these
systems.
Information provided by the vendor in this section of the TDP may
be duplicative of information required by other sections. Vendors may
cross reference to information provided in other sections provided that
the means used provides a clear mapping to the requirements of this
section.
Information submitted by the vendor shall be used by the test
authority to assist in developing and executing the system
qualification test plan. The Security Specification shall contain the
sections identified below.
2.6.1 Access Control Policy
The vendor shall specify the features and capabilities of the
access control policy recommended to purchasing jurisdictions to
provide effective voting system security to meet the specific
requirements of Volume I, Section 6.2.1. The access control policy
shall address the general features and capabilities and individual
access privileges indicated in Volume I, Section 6.2.1.
2.6.2 Access Control Measures
The vendor shall provide a detailed description of all system
access control measures and mandatory procedures designed to permit
access to system states in accordance with the access policy, and to
prevent all other types of access to meet the specific requirements of
Volume I, Section 6.2.2.
The vendor also shall define and provide a detailed description of
the methods used to preclude unauthorized access to the access control
capabilities of the system itself.
2.6.3 Equipment and Data Security
The vendor shall provide a detailed description of system
capabilities and mandatory procedures for purchasing jurisdictions to
prevent disruption of the voting process and corruption of voting data
to meet the specific requirements of Volume I, Section 6.3 of the
Standards. This information shall address measures for polling place
security and central count location security.
2.6.4 Software Installation
The vendor shall provide a detailed description of the system
capabilities and mandatory procedures for purchasing jurisdictions to
ensure secure software (including firmware) installation to meet the
specific requirements of Volume I, Section 6.4 of the Standards. This
information shall address software installation for all system
components.
2.6.5 Telecommunications and Data Transmission Security
The vendor shall provide a detailed description of the system
capabilities and mandatory procedures for purchasing jurisdictions to
ensure secure data transmission to meet the specific requirements of
Volume I, Section 6.5:
a. For all systems, this information shall address access control,
and prevention of data interception; and
b. For systems that use public communications networks as defined
in Volume I Section 5, this information shall also include:

[[Continued on page 19025]]

From the Federal Register Online via GPO Access [wais.access.gpo.gov]
]

[[pp. 19025-19052]] Technical Guidelines Development Committee (TGDC); Initial
Report: Voluntary Voting System Guidelines Version I

[[Continued from page 19024]]

[[Page 19025]]

(1) Capabilities used to provide protection against threats to
third party products and services;
(2) Policies and processes used by the vendor to ensure that such
protection is updated to remain effective over time;
(3) Policies and procedures used by the vendor to ensure that
current versions of such capabilities are distributed to user
jurisdictions and are installed effectively by the jurisdiction;
(4) A detailed description of the system capabilities and
procedures to be employed by the jurisdiction to diagnose the
occurrence of a denial of service attack, to use an alternate method of
voting, to determine when it is appropriate to resume voting over the
network, and to consolidate votes cast using the alternate method;
(5) A detailed description of all activities to be performed in
setting up the system for operation that are mandatory to ensure
effective system security, including testing of security before an
election; and
(6) A detailed description of all activities that should be
prohibited during system setup and during the timeframe for voting
operations, including both the hours when polls are open and when polls
are closed.
2.6.6 Other Elements of an Effective Security Program
The vendor shall provide a detailed description of the following
additional procedures required for use by the purchasing jurisdiction:
a. Administrative and management controls for the voting system and
election management, including access controls;
b. Internal security procedures, including operating procedures for
maintaining the security of the software for each system function and
operating mode;
c. Adherence to, and enforcement of, operational procedures (e.g.,
effective password management);
d. Physical facilities and arrangements; and
e. Organizational responsibilities and personnel screening.
This documentation shall be prepared such that these requirements
can be integrated by the jurisdiction into local administrative and
operating procedures.

2.7 System Test and Verification Specification

The vendor shall provide test and verification specifications for:
a. Development test specifications; and
b. Qualification test specifications.
2.7.1 Development Test Specifications
The vendor shall describe the plans, procedures, and data used
during software development and system integration to verify system
logic correctness, data quality, and security. This description shall
include:
a. Test identification and design, including:
(1) Test structure;
(2) Test sequence or progression; and
(3) Test conditions;
a. Standard test procedures, including any assumptions or
constraints;
b. Special purpose test procedures including any assumptions or
constraints;
c. Test data; including the data source, whether it is real or
simulated, and how test data is controlled;
d. Expected test results; and
e. Criteria for evaluating test results.
Additional details for these requirements are provided by MIL-STD-
498, Software Test Plan (STP) and Software Test Description (STD). In
the event that test data is not available, the ITA shall design test
cases and procedures equivalent to those ordinarily used during product
verification.
2.7.2 Qualification Test Specifications
The vendor shall provide specifications for verification and
validation of overall software performance. These specifications shall
cover:
a. Control and data input/output;
b. Acceptance criteria;
c. Processing accuracy;
d. Data quality assessment and maintenance;
e. Ballot interpretation logic;
f. Exception handling;
g. Security; and
h. Production of audit trails and statistical data.
The specifications shall identify procedures for assessing and
demonstrating the suitability of the software for elections use.
2.8 System Operations Procedures
This documentation shall provide all information necessary for
system use by all personnel who support pre-election and election
preparation, polling place activities and central counting activities,
as applicable, with regard to all system functions and operations
identified in Section 2.3 above. The nature of the instructions for
operating personnel will depend upon the overall system design and
required skill level of system operations support personnel.
The system operations procedures shall contain all information that
is required for the preparation of detailed system operating
procedures, and for operator training, including the sections listed
below:
2.8.1 Introduction
The vendor shall provide a summary of system operating functions
and modes, in sufficient detail to permit understanding of the system's
capabilities and constraints. The roles of operating personnel shall be
identified and related to the operating modes of the system. Decision
criteria and conditional operator functions (such as error and failure
recovery actions) shall be described.
The vendor shall also list all reference and supporting documents
pertaining to the use of the system during elections operations.
2.8.2 Operational Environment
The vendor shall describe the system environment, and the interface
between the user or operator and the system. The vendor shall identify
all facilities, furnishings, fixtures, and utilities that will be
required for equipment operations, including equipment that operates at
the:
a. Polling place;
b. Central count facility; and
c. Other locations.
2.8.3 System Installation and Test Specification
The vendor shall provide specifications for validation of system
installation, acceptance, and readiness. These specifications shall
address all components of the system and all locations of installation
(e.g., polling place central count facility), and shall address all
elements of system functionality and operations identified in Section
2.3 above, including:
a. Pre-voting functions;
b. Voting functions;
c. Post-voting functions; and
d. General capabilities.
These specifications also serve to provide guidance to the
procuring agency in developing its acceptance test plan and procedure
according to the agency's contract provisions, and the election laws of
the state.
2.8.4 Operational Features
The vendor shall provide documentation of system operating features
that meets the following requirements:
a. Provides a detailed description of all input, output, control,
and display features accessible to the operator or voter;
b. Provide examples of simulated interactions in order to
facilitate

[[Page 19026]]

understanding of the system and its capabilities;
c. Provide sample data formats and output reports; and
d. Illustrate and describe all status indicators and information
messages.
2.8.5 Operating Procedures
The vendor shall provide documentation of system operating
procedures that meets the following requirements:
a. Provides a detailed description of procedures required to
initiate, control, and verify proper system operation;
b. Provides procedures that clearly enable the operator to assess
the correct flow of system functions (as evidenced by system-generated
status and information messages);
c. Provides procedures that clearly enable the operator to
intervene the system operations to recover from an abnormal system
state;
d. Defines and illustrates the procedures and system prompts for
situations where operator intervention is required to load, initialize,
and start the system;
e. Define and illustrate procedures to enable and control the
external interface to the system operating environment if supporting
hardware and software are involved (such information shall be provided
for the interaction of the system with other data processing systems or
data interchange protocols as well);
f. Provide administrative procedures and off-line operator duties
(if any) if they relate to the initiation or termination of system
operations, to the assessment of system status, or to the development
of an audit trail;
g. To support successful ballot and program installation and
control by election officials, provide a detailed work plan or other
form of documentation providing a schedule and steps for the software
and ballot installation, which includes a table outlining the key
dates, events and deliverables; and
h. To support diagnostic testing, specify diagnostic tests that may
be employed to identify problems in the system, verify the correction
of maintenance problems; and isolate and diagnose faults from various
systems states.
2.8.6 Operations Support
The vendor shall provide documentation of system operating
procedures that meets the following requirements:
a. Defines the procedures required to support system acquisition,
installation, and readiness testing (these procedures may be provided
by reference, if they are contained either in the system hardware
specifications, or in other vendor documentation provided to the ITA
and to system users); and
b. Describe procedures for providing technical support, system
maintenance and correction of defects, and for incorporating hardware
upgrades and new software releases.
2.8.7 Appendices
The vendor may provide descriptive material and data supplementing
the various sections of the body of the System Operations Manual. The
content and arrangement of appendices shall be at the discretion of the
vendor. Topics recommended for discussion include:
a. Glossary: A listing and brief definition of all terms that may
be unfamiliar to persons not trained in either voting systems or
computer operations;
b. References: A list of references to all vendor documents and to
other sources related to operation of the system;
c. Detailed Examples: Detailed scenarios that outline correct
system responses to faulty operator input. Alternative procedures may
be specified depending on the system state; and
d. Manufacturer's Recommended Security Procedures: This appendix
shall contain the security procedures that are to be executed by the
system operator.

2.9 System Maintenance Procedures

The system maintenance procedures shall provide information in
sufficient detail to support election workers, data personnel, or
maintenance personnel in the adjustment or removal and replacement of
components or modules in the field. Technical documentation needed
solely to support the repair of defective components or modules
ordinarily done by the manufacturer or software developer is not
required.
Recommended service actions to correct malfunctions or problems
shall be discussed, along with personnel and expertise required to
repair and maintain the system; and equipment, materials, and
facilities needed for proper maintenance. This manual shall include the
sections listed below.
2.9.1 Introduction
The vendor shall describe the structure and function of the
equipment (and related software) for election preparation, programming,
vote recording, tabulation, and reporting in sufficient detail to
provide an overview of the system for maintenance, and for
identification of faulty hardware or software. The description shall
include a theory of operation that fully describes such items as:
a. The electrical and mechanical functions of the equipment;
b. How the processes of ballot handling and reading are performed
(paper-based systems);
c. How vote selection and casting of the ballot are performed (DRE
systems);
d. How transmission of data over a network are performed (DRE
systems, where applicable);
e. How data are handled in the processor and memory units;
f. How data output is initiated and controlled;
g. How power is converted or conditioned; and
h. How test and diagnostic information is acquired and used.
2.9.2 Maintenance Procedures
The vendor shall describe preventive and corrective maintenance
procedures for hardware and software.
2.9.2.1 Preventive Maintenance Procedures
The vendor shall identify and describe:
a. All required and recommended preventive maintenance tasks,
including software tasks such as software backup, database performance
analysis, and database tuning;
b. Number and skill levels of personnel required for each task;
c. Parts, supplies, special maintenance equipment, software tools,
or other resources needed for maintenance; and
d. Any maintenance tasks that must be coordinated with the vendor
or a third party (such as coordination that may be needed for off-the-
shelf items used in the system).
2.9.2.2 Corrective Maintenance Procedures
The vendor shall provide fault detection, fault isolation,
correction procedures, and logic diagrams for all operational
abnormalities identified by design analysis and operating experience.
The vendor shall identify specific procedures to be used in
diagnosing and correcting problems in the system hardware (or user-
controlled software). Descriptions shall include:
a. Steps to replace failed or deficient equipment;
b. Steps to correct deficiencies or faulty operations in software;
c. Modifications that are necessary to coordinate any modified or
upgraded software with other software modules;

[[Page 19027]]

d. The number and skill levels of personnel needed to accomplish
each procedure;
e. Special maintenance equipment, parts, supplies, or other
resources needed to accomplish each procedure; and
f. Any coordination required with the vendor, or other party for
off the shelf items.
2.9.3 Maintenance Equipment
The vendor shall identify and describe any special purpose tests or
maintenance equipment recommended for fault isolation and diagnostic
purposes.
2.9.4 Parts and Materials
Vendors shall provide detailed documentation of parts and materials
needed to operate and maintain the system. Additional requirements
apply for paper-based systems.
2.9.4.1 Common Standards
The vendor shall provide a complete list of approved parts and
materials needed for maintenance. This list shall contain sufficient
descriptive information to identify all parts by:
a. Type;
b. Size;
c. Value or range;
d. Manufacturer's designation;
e. Individual quantities needed; and
f. Sources from which they may be obtained.
2.9.4.2 Paper-Based Systems
For marking devices manufactured by multiple external sources, the
vendor shall provide a listing of sources and model numbers that are
compatible with the system.
The TDP shall specify the required paper stock, size, shape,
opacity, color, watermarks, field layout, orientation, size and style
of printing, size and location of punch or mark fields used for vote
response fields and to identify unique ballot formats, placement of
alignment marks, ink for printing, and folding and bleed-through
limitations for preparation of ballots that are compatible with the
system.
2.9.5 Maintenance Facilities and Support
The vendor shall identify all facilities, furnishings, fixtures,
and utilities that will be required for equipment maintenance. In
addition, vendors shall specify the assumptions made with regard to any
parameters that impact the mean time to repair. These factors shall
include at a minimum:
a. Recommended number and locations of spare devices or components
to be kept on hand for repair purposes during periods of system
operation;
b. Recommended number and locations of qualified maintenance
personnel who need to be available to support repair calls during
system operation; and
c. Organizational affiliation (i.e., jurisdiction, vendor) of
qualified maintenance personnel.
2.9.6 Appendices
The vendor may provide descriptive material and data supplementing
the various sections of the body of the System Maintenance Manual. The
content and arrangement of appendices shall be at the discretion of the
vendor. Topics recommended for amplification or treatment in appendix
include:
a. Glossary: A listing and brief definition of all terms that may
be unfamiliar to persons not trained in either voting systems or
computer maintenance;
b. References: A list of references to all vendor documents and
other sources related to maintenance of the system;
c. Detailed Examples: Detailed scenarios that outline correct
system responses to every conceivable faulty operator input.
Alternative procedures may be specified depending on the system state;
and
d. Maintenance and Security Procedures: This appendix shall contain
technical illustrations and schematic representations of electronic
circuits unique to the system.

2.10 Personnel Deployment and Training Requirements

The vendor shall describe the personnel resources and training
required for a jurisdiction to operate and maintain the system.
2.10.1 Personnel
The vendor shall specify the number of personnel and skill level
required to perform each of the following functions:
a. Pre-election or election preparation functions (e.g., entering
an election, race and candidate information; designing a ballot;
generating pre-election reports;
b. System operations for voting system functions performed at the
polling place;
c. System operations for voting system functions performed at the
central count facility;
d. Preventive maintenance tasks;
e. Diagnosis of faulty hardware or software;
f. Corrective maintenance tasks; and
g. Testing to verify the correction of problems.
A description shall be presented of which functions may be carried
out by user personnel, and those that must be performed by vendor
personnel.
2.10.2 Training
The vendor shall specify requirements for the orientation and
training of the following personnel:
a. Poll workers supporting polling place operations;
b. System support personnel involved in election programming;
c. User system maintenance technicians;
d. Network/system administration personnel (if a network is used);
e. Data personnel; and
f. Vendor personnel.

2.11 Configuration Management Plan

Vendors shall submit a Configuration Management Plan that addresses
the configuration management requirements of Volume I, Section 8 of the
Standards.
This plan shall describe all policies, processes and procedures
employed by the vendor to carry out these requirements. Information
submitted by the vendor shall be used by the test authority to assist
in developing and executing the system qualification test plan. This
information is particularly important to support the design of test
plans for system modifications. A well-organized, robust and detailed
Configuration Management Plan will enable the test authority to more
readily determine the nature and scope of tests needed to fully test
the modifications. The Configuration Management Plan shall contain the
sections identified below.
2.11.1 Configuration Management Policy
The vendor shall provide a description of its organizational
policies for configuration management, addressing the specific
requirements of Volume I, Section 8.3 of the Standards. These
requirements pertain to:
a. Scope and nature of configuration management program activities;
and
b. Breadth of application of vendor's policy and practices to the
voting system.
2.11.2 Configuration Identification
The vendor shall provide a description of the procedures and naming
conventions used to address the specific requirements of Volume I,
Section 8.4. These requirements pertain to:
a. Classifying configuration items into categories and
subcategories;
b. Uniquely numbering or otherwise identifying configuration items;
and
c. Naming configuration items.

[[Page 19028]]

2.11.3 Baseline, Promotion, and Demotion Procedures
The vendor shall provide a description of the procedures and naming
conventions used to address the specific requirements of Volume I,
Section 8.5 of the Standards. These requirements pertain to:
a. Establishing a particular instance of a system component as the
starting baseline;
b. Promoting subsequent instances of a component to baseline
throughout the system development process for the first complete
version of the system submitted for qualification testing; and
c. Promoting subsequent instances of a component to baseline status
as the component is maintained throughout its life cycle.
2.11.4 Configuration Control Procedures
The vendor shall provide a description of the procedures used by
the vendor to approve and implement changes to a configuration item to
prevent unauthorized additions, changes, or deletions to address the
specific requirements of Volume I, Section 8.6 of the Standards. These
requirements pertain to:
a. Developing and maintaining internally developed items;
b. Developing and maintaining third-party items;
c. Resolve internally identified defects; and
d. Resolve externally identified and reported defects.
2.11.5 Release Process
The vendor shall provide a description of the contents of a system
release, and the procedures and related conventions by which the vendor
installs, transfers, or migrates the system to ITAs and customers to
address the specific requirements of Volume I, Section 8.7 of the
Standards. These requirements pertain to:
a. A first release of the system to an ITA;
b. A subsequent maintenance or upgrade release of a system, or
particular components, to an ITA;
c. The initial delivery and installation of the system to a
customer; and
d. A subsequent maintenance or upgrade release of a system, or
particular components, to a customer.
2.11.6 Configuration Audits
The vendor shall provide a description of the procedures and
related conventions for the two audits required by Volume I, Section
8.8 of the Standards. These requirements pertain to:
a. Physical configuration audit that verifies the voting system
components submitted for qualification to the vendor's technical
documentation; and
b. Functional configuration audit that verifies the system performs
all the functions described in the system documentation.
2.11.7 Configuration Management Resources
The vendor shall provide a description of the procedures and
related conventions for maintaining information about configuration
management tools required by Volume I, Section 8.9 of the Standards.
These requirements pertain to information regarding:
a. Specific tools used, current version, and operating environment;
b. Physical location of the tools, including designation of
computer directories and files; and
c. Procedures and training materials for using the tools.

2.12 Quality Assurance Program

Vendors shall submit a Quality Assurance Program that addresses the
quality assurance requirements of Volume I, Section 7. This plan shall
describe all policies, processes and procedures employed by the vendor
to ensure the overall quality of the system for its initial development
and release and for subsequent modifications and releases. This
information is particularly important to support the design of test
plans by the test authority. A well-organized, robust and detailed
Quality Assurance Program will enable the test authority to more
readily determine the nature and scope of tests needed to test the
system appropriately. The Quality Assurance Program shall, at a
minimum, address the topics indicated below.
2.12.1 Quality Assurance Policy
The vendor shall provide a description of its organizational
policies for quality assurance, including:
a. Scope and nature of QA activities; and
b. Breadth of application of vendor's policy and practices to the
voting system.
2.12.2 Parts and Materials Special Tests and Examinations
The vendor shall provide a description of its practices for parts
and materials tests and examinations that meet the requirements of
Volume I, Section 7.3 of the Standards.
2.12.3 Quality Conformance Inspections
The vendor shall provide a description of its practices for quality
conformance inspections that meet the requirements of Volume I, Section
7.4 of the Standards. For each test performed, the record of tests
provided shall include:
a. Test location;
b. Test date;
c. Individual who conducted the test; and
d. Test outcomes.
2.12.4 Documentation
The vendor shall provide a description of its practices for
documentation of the system and system development process that meet
the requirements of Volume I, Section 7.5 of the Standards.

2.13 System Change Notes

Vendors submitting a system for testing that has been tested
previously by the test authority and issued a qualification number
shall submit system change notes. These will be used by the test
authority to assist in developing and executing the test plan for the
modified system. The system change notes shall include the following
information:
a. Summary description of the nature and scope of the changes, and
reasons for each changes;
b. A listing of the specific changes made, citing the specific
system configuration items changed and providing detailed references to
the sections of documentation changed;
c. The specific sections of the documentation that are changed (or
complete revised documents, if more suitable to address a large number
of changes);
d. Documentation of the test plan and procedures executed by the
vendor for testing the individual changes and the system as a whole,
and records of test results.

Volume II, Section 3

Table of Contents

3 Functionality Testing

3.1 Scope
3.2 Breadth of Functionality Testing
3.2.1 Basic Functionality Testing Requirements
3.2.2 Variation of System Functionality Testing to Reflect
Voting System Technologies and Configurations
3.2.3 Variation of System Functionality Testing to Reflect
Additional Voting System Capabilities
3.2.4 Variation of System Functionality Testing to Reflect
Voting Systems That Incorporate Previously Tested Functionality
3.3 General Test Sequence
3.3.1 Functionality Testing in Parallel With Hardware Testing
for Precinct Count Systems

[[Page 19029]]

3.3.2 Functionality Testing in Parallel with Hardware Testing
for Central Count Systems
3.4 Functionality Testing for Accessibility
3.5 Functionality Testing for Systems That Operate on Personal
Computers

3 Functionality Testing

3.1 Scope

This section contains a description of the testing to be performed
by the ITAs to confirm the functional capabilities of a voting system
submitted for qualification. It describes the scope and basis for
functionality testing, outlines the general sequence of tests within
the overall test process, and provides guidance on testing for
accessibility.

3.2 Breadth of Functionality Testing

In order to best compliment the diversity of the voting systems
industry, the qualification testing process is not rigidly defined.
Although there are basic functionality testing requirements, additions
or variations in testing are appropriate in order to complement the
system's use of specific technologies and configurations, the system
capabilities, and the outcomes of previous testing.
3.2.1 Basic Functionality Testing Requirements
ITAs shall design and perform procedures to test a voting system
against the functional requirements outlined in Volume I, Section 2.
Tests procedures shall be designed and performed by the ITA that
address:
a. Overall system capabilities;
b. Pre-voting functions;
c. Voting functions;
d. Post-voting functions;
e. System maintenance; and
f. Transportation and storage.
The specific procedures to be used shall be identified in the
Qualification Test Plan prepared by the ITA. These procedures may
replicate testing performed by the vendor and documented in the
vendor's TDP, but shall not rely on vendor testing as a substitute for
functionality testing performed by the ITA.
Recognizing variations in system design and the technologies
employed by different vendors, the ITAs shall design test procedures
that account for such variations and reflect the system-specific
functional capabilities in Volume I, Section 2.
3.2.2 Variation of System Functionality Testing to Reflect Voting
System Technologies and Configurations
Voting systems are not designed according to a standard design
template. Instead, system design reflects the vendor's selections from
a variety of technologies and design configurations. Such variation is
recognized in the definitions of voting systems in Volume I, Section 1,
and serves as the basis for delineating various functional capability
requirements.
Functional capabilities will vary according to the relative
complexity of a system and the manner in which the system integrates
various technologies. Therefore, the testing procedure designed and
performed by the ITA for a particular system shall reflect the specific
technologies and design configurations used by that system.
3.2.3 Variation of System Functionality Testing to Reflect Additional
Voting System Capabilities
The requirements for voting system functionality provided by Volume
I, Section 2 reflect a minimum set of capabilities. Vendors may, and
often do, provide additional capabilities in systems that are submitted
for qualification testing in order to respond to the requirements of
individual states. These additional capabilities shall be identified by
the vendor within the TDP as described in Volume II, Section 2. Based
on this information, ITAs shall design and perform system functionality
testing for additional functional capabilities as well as the
capabilities required by Volume I, Section 2 of the Standards.
3.2.4 Variation of System Functionality Testing to Reflect Voting
Systems That Incorporate Previously Tested Functionality
The required functional capabilities of voting systems defined in
Volume I, Section 2 reflect a broad range of system functionality
needed to support the full life cycle of an election, including post
election activities. Many systems submitted for qualification testing
are designed to address this scope, and are tested accordingly.
However, some new systems seek qualification using a combination of
new subsystems or system components interfaced with the components of a
previously qualified system. For example, a vendor can submit a voting
system for qualification testing that has a new DRE voting device, but
that integrates the election management component from a previously
qualified system.
In this situation, the vendor is strongly encouraged to identify in
its TDP the functional capabilities supported by new subsystems/
components and those supported by subsystems/components taken from a
previously qualified system. The vendor is also encouraged to indicate
in its system design documentation and configuration management records
the scope and nature of any modifications made to the reused subsystems
or components. Following these suggestions will assist the ITA in
developing efficient test procedures that rely in part on the results
of testing of the previously qualified subsystems or components.
In this situation the ITA may design and perform a test procedure
that draws on the results of testing performed previously on reused
subsystems or components. However, the scope of testing shall include,
irrespective of previous testing, certain functionality tests:
a. All functionality performed by new subsystems/modules;
b. All functionality performed by modified subsystems/modules;
c. Functionality that is accomplished using any interfaces to new
modules, or that shares inputs or outputs from new modules;
d. All functionality related to vote tabulation and election
results reporting; and
e. All functionality related to audit trail maintenance.

3.3 General Test Sequence

There is no required sequence for performing the system
qualification tests. For a system not previously qualified, the ITA may
perform tests using generic test ballots, and schedule the tests in a
convenient order, provided that prerequisite conditions for each test
have been satisfied before the test is initiated.
Regardless of the sequence of testing used, the full qualification
testing process shall include functionality testing for all system
functions of a voting system, minus the exceptions noted in Section
3.2. Generally, in depth functionality testing will follow testing of
the systems hardware and the source code review of the system's
software. ITAs will usually conduct functionality testing as an
integral element of system level integration testing described in
Volume II, Section 6.
Some functionality tests for the voting functions defined in Volume
I, Section 2.4 and 2.5 may be performed as an integral part of hardware
testing, enabling a more efficient testing process. Ballots processed
and counted during hardware operating tests for precinct count and
central count systems may serve to satisfy part of the functionality
testing provided that the ballots were cast using a test procedure

[[Page 19030]]

that is equivalent to the procedures indicated below.
3.3.1 Functionality Testing in Parallel With Hardware Testing for
Precinct Count Systems
For testing voting functions defined in Volume I, Sections 2.4 and
2.5, the following procedures shall be performed during the
functionality tests of voting equipment and precinct counting
equipment.
a. The procedure to prepare election programs shall:
(1) Verify resident firmware, if any;
(2) Prepare software (including firmware) to simulate all ballot
format and logic options for which the system will be used;
(3) Verify program memory device content; and
(4) Obtain and design test ballots with formats and voting patterns
sufficient to verify performance of the test election programs.
b. The procedures to program precinct ballot counters shall:
(1) Install program and data memory devices, or verify presence if
resident; and
(2) Verify operational status of hardware as in Volume II, Section
4.
c. The procedures to simulate opening of the polls shall:
(1) Perform procedures required to prepare hardware for election
operations;
(2) Obtain ``zero'' printout or other evidence that data memory has
been cleared;
(3) Verify audit record of pre-election operations; and
(4) Perform procedure required to open the polling place and enable
ballot counting.
d. The procedure to simulate counting ballots shall cast test
ballots in a number sufficient to demonstrate proper processing, error
handling, and generation of audit data as specified in Volume I,
Sections 2 and 4.
e. The procedure to simulate closing of polls shall:
(1) Perform hardware operations required to disable ballot counting
and close the polls;
(2) Obtain data reports and verify correctness; and
(3) Obtain audit log and verify correctness.
They need not be performed in the sequence listed, provided the
necessary precondition of each procedure has been met.
3.3.2 Functionality Testing in Parallel With Hardware Testing for
Central Count Systems
For testing voting functions defined in Volume I, Sections 2.4 and
2.5, the following procedures shall be performed during the functional
tests.
a. The procedure to prepare election programs shall:
(1) Verify resident firmware, if any;
(2) Prepare software (including firmware) to simulate all ballot
format and logic options for which the system will be used, and to
enable simulation of counting ballots from at least 10 polling places
or precincts;
(3) Verify program memory device content; and
(4) Procure test ballots with formats, voting patterns, and format
identifications sufficient to verify performance of the test election
programs;
b. The procedure to simulate counting ballots shall count test
ballots in a number sufficient to demonstrate proper processing, error
handling, and generation of audit data as specified in Volume I,
Sections 2 and 4; and
c. The procedure to simulate election reports shall:
(1) Obtain reports at polling places or precinct level;
(2) Obtain consolidated reports;
(3) Provide query access, if this is a feature of the system;
(4) Verify correctness of all reports and queries; and
(5) Obtain audit log and verify correctness.
They need not be performed in the sequence listed, provided the
necessary preconditions of each procedure have been met.

3.4 Functionality Testing for Accessibility

As indicated in Volume I, Section 2.2.7, voting systems shall
provide accessibility to individuals with disabilities, meeting the
specific requirements of this Section. ITAs shall design and perform
test procedures that verify conformance with each of these
requirements.

3.5 Functionality Testing for Systems That Operate on Personal
Computers

For systems intended to use non-standard voting devices, such as a
personal computer, provided by the local jurisdiction, ITAs shall
conduct functionality tests using hardware provided by the vendor that
meets the minimum configuration specifications defined by the vendor.
Volume II, Section 4, provides additional information on hardware
to be used to conduct functionality testing of such voting devices, as
well as hardware to be used to conduct security testing and other forms
of testing.

Volume II, Section 4

Table of Contents

4 Hardware Testing

4.1 Scope
4.2 Basis of Hardware Testing
4.2.1 Testing Focus and Applicability
4.2.2 Hardware Provided by Vendor
4.3 Test Conditions
4.4 Test Log Data Requirements
4.5 Test Fixtures
4.6 Non-operating Environmental Tests
4.6.1 General
4.6.1.1 Pretest Data
4.6.1.2 Preparation for Test
4.6.1.3 Mechanical Inspection and Repair
4.6.1.4 Electrical Inspection and Adjustment
4.6.1.5 Operational Status Check
4.6.1.6 Failure Criteria
4.6.2 Bench Handling Test
4.6.2.1 Applicability
4.6.2.2 Procedure
4.6.3 Vibration Test
4.6.3.1 Applicability
4.6.3.2 Procedure
4.6.4 Low Temperature Test
4.6.4.1 Applicability
4.6.4.2 Procedure
4.6.5 High Temperature Test
4.6.5.1 Applicability
4.6.5.2 Procedure
4.6.6 Humidity Test
4.6.6.1 Applicability
4.6.6.2 Procedure
4.7 Environmental Tests, Operating
4.7.1 Temperature and Power Variation Tests
4.7.1.1 Data Accuracy
4.7.2 Maintainability Test
4.7.3 Reliability Test
4.7.4 Availability Test
4.8 Other Environmental Tests
4.8.1 Power Disturbance
4.8.2 Electromagnetic Radiation
4.8.3 Electrostatic Disruption
4.8.4 Electromagnetic Susceptibility
4.8.5 Electrical Fast Transient
4.8.6 Lightning Surge
4.8.7 Conducted RF Immunity
4.8.8 Magnetic Fields Immunity

4 Hardware Testing

4.1 Scope

This section contains a description of the testing to be performed
by the ITAs to confirm the proper functioning of the hardware
components of a voting system submitted for qualification testing. It
describes the scope and basis for functionality testing, required test
conditions for conducting hardware testing, guidance for the use of
test fixtures, test log data requirements, and test practices for
specific non-operating and operating environmental tests.

4.2 Basis of Hardware Testing

This section addresses the focus and applicability of hardware
testing, and specifies the vendor's obligations to produce hardware to
conduct such tests.

[[Page 19031]]

4.2.1 Testing Focus and Applicability
ITAs shall design and perform procedures that test the voting
system hardware requirements identified in Volume I, Section 3. Test
procedures shall be designed and performed by the ITA for both
operating and non-operating environmental tests:
Operating environmental tests apply to the entire system,
including hardware components that are used as part of the voting
system telecommunications capability; and
Non-operating tests apply to those elements of the system
that are intended for use at poll site voting locations, such as voting
machines and precinct counters. These tests address environmental
conditions that may be encountered by the voting system hardware at the
voting location itself, or while in storage or transit to or from the
poll site.
Additionally, compatibility of this equipment with the voting
system environment shall be determined through functional tests
integrating the standard product with the remainder of the system.
All hardware components custom-designed for election use shall be
tested in accordance with the applicable procedures contained in this
section. Unmodified COTS hardware will not be subject to all tests.
Generally such equipment has been designed to rigorous industrial
standards and has been in wide use, permitting an evaluation of its
performance history. To enable reduced testing of such equipment,
vendors shall provide the manufacturers specifications and evidence
that the equipment has been tested to the equivalent of the Standards.
The specific testing procedures to be used shall be identified in
the Qualification Test Plan prepared by the ITA. These procedures may
replicate testing performed by the vendor and documented in the
vendor's TDP, but shall not rely on vendor testing as a substitute for
hardware testing performed by the ITA.
4.2.2 Hardware Provided by Vendor
The hardware submitted for qualification testing shall be
equivalent, in form and function, to the actual production versions of
the hardware units. Engineering or developmental prototypes are not
acceptable unless the vendor can show that the equipment to be tested
is equivalent to standard production units in both performance and
construction.

4.3 Test Conditions

Qualification tests may be performed in any facility capable of
supporting the test environment. Preparation for testing, arrangement
of equipment, verification of equipment status, and the execution of
procedures shall be witnessed by at least one independent, qualified
observer who shall certify that all test and data acquisition
requirements have been satisfied.
When a test is to be performed at ``standard'' or ``ambient''
conditions, this requirement shall refer to a nominal laboratory
environment at prevailing atmospheric pressure and relative humidity.
Otherwise, all tests shall be performed at the required temperature
and electrical supply voltage, regulated within the following
tolerances:
a. Temperature of +/-4 degrees F; and
b. Electrical supply voltage +/-2 VAC.

4.4 Test Log Data Requirements

The ITA shall maintain a test log of the procedure employed. This
log shall identify the system and equipment by model and serial number.
Test environment conditions shall be noted.
In the event that the ITA deems it necessary to deviate from
requirements pertaining to the test environment, the equipment
arrangement and method of operation, the specified test procedure, or
the provision of test instrumentation and facilities, the deviation
shall be recorded in the test log. A discussion of the reasons for the
deviation and the effect of the deviation on the validity of the test
procedure shall also be provided.

4.5 Test Fixtures

The use of test fixtures or ancillary devices to facilitate
hardware qualification testing is encouraged. These fixtures and
devices may include arrangements for automating the operation of voting
devices and the acquisition of test data.
The use of a fixture to ensure correctness in casting ballots by
hand is recommended. Such a fixture may consist of a template, with
apertures in the desired location, so that selections may be made
rapidly. Such a template will eliminate or greatly minimize errors in
activating test ballot patterns, while reducing the amount of time
required to cast a test ballot.
For systems that use a light source as a means of detecting voter
selections, the generation of a suitable optical signal by an external
device is acceptable. For systems that rely on the physical activation
of a switch, a mechanical fixture with suitable motion generators is
acceptable.
To speed up the process of testing and to eliminate human error in
casting test ballots the tests may use a simulation device with
appropriate software. Such simulation is recommended if it covers all
voting data detection and control paths that are used in casting an
actual ballot. In the event that only partial simulation is achieved,
then an independent method and test procedure must be used to validate
the proper operation of those portions of the system not tested by the
simulator.
If the vendor provides a means of simulating the casting of
ballots, the simulation device is subject to the same performance,
reliability, and quality requirements that apply to the voting device
itself so as not to contribute errors to the test processes.

4.6 Non-Operating Environmental Tests

This section addresses a range of tests for voting machines and
precinct counters, as such devices are stored between elections and are
transported between the storage facility and polling site.
4.6.1 General
Environmental tests of non-operating equipment are intended to
simulate exposure to physical shock and vibration associated with
handling and transportation of voting equipment and precinct counters
between a jurisdiction's storage facility and precinct polling site.
These tests additionally simulate the temperature and humidity
conditions that may be encountered during storage in an uncontrolled
warehouse environment or precinct environment. The procedures and
conditions of these tests correspond generally to those of MIL-STD-
810D, ``Environmental Test Methods and Engineering Guidelines,'' 19
July 1983. In most cases, the severity of the test conditions has been
reduced to reflect commercial, rather than military, practice.
Systems exclusively designed with system-level COTS hardware whose
configuration has not been modified in any manner and are not subjected
to this segment of hardware testing. Systems made up of individual COTS
components such as hard drives, motherboards, and monitors that have
been packaged to build a voting machine or other device will be
required to undergo the hardware testing.
Prior to each test, the equipment shall be shown to be operational
by means of

[[Page 19032]]

the procedure contained in Subsection 4.6.1.5. The equipment may then
be prepared as if for actual transportation or storage, and subjected
to appropriate test procedures outlined. After each procedure has been
completed, the equipment status will again be verified as in Subsection
4.6.1.5.
The following requirements for equipment preparation, functional
tests, and inspections shall apply to each of the non-operating test
procedures.
4.6.1.1 Pretest Data
The test technician shall verify that the equipment is capable of
normal operation. Equipment identification, environmental conditions,
equipment configuration, test instrumentation, operator tasks, time-of-
day or test time, and test results shall be recorded.
4.6.1.2 Preparation for Test
The equipment shall be prepared as for the expected non-operating
use, as noted below. When preparation for transport between the storage
site and the polling place is required, the equipment shall be prepared
with any protective enclosures or internal restraints that the vendor
specifies for such transport. When preparation for storage is required,
the equipment shall be prepared using any protective enclosures or
internal restraints that the vendor specifies for storage.
4.6.1.3 Mechanical Inspection and Repair
After the test has been completed, the devices shall be removed
from their containers, and any internal restraints shall be removed.
The exterior and interior of the devices shall be inspected for
evidence of mechanical damage, failure, or dislocation of internal
components. Devices shall be adjusted or repaired, if necessary.
4.6.1.4 Electrical Inspection and Adjustment
After completion of the mechanical inspection and repair, routine
electrical maintenance and adjustment may be performed, according to
the manufacturer's standard procedure.
4.6.1.5 Operational Status Check
When all tests, inspections, repairs, and adjustments have been
completed, normal operation shall be verified by conducting an
operational status check.
During this process, all equipment shall be operated in a manner
and environmental conditions that simulate election use to verify the
functional status of the system. Prior to the conduct of each of the
environmental hardware non-operating tests, a supplemental test shall
be made to determine that the operational state of the equipment is
within acceptable performance limits.
The following procedures shall be followed to verify the equipment
status:
Step 1: Arrange the system for normal operation.
Step 2: Turn on power, and allow the system to reach recommended
operating temperature.
Step 3: Perform any servicing, and make any adjustments necessary,
to achieve operational status.
Step 4: Operate the equipment in all modes, demonstrating all
functions and features that would be used during election operations.
Step 5: Verify that all system functions have been correctly
executed.
4.6.1.6 Failure Criteria
Upon completion of each non-operating test, the system hardware
shall be subject to functional testing to verify continued operability.
If any portion of the voting machine or precinct counter hardware fails
to remain fully functional, the testing will be suspended until the
failure is identified and corrected by the vendor. The system will then
be subject to a retest.
4.6.2 Bench Handling Test
The bench handling test simulates stresses faced during maintenance
and repair of voting machines and ballot counters.
4.6.2.1 Applicability
All systems and components, regardless of type, shall meet the
requirements of this test. This test is equivalent to the procedure of
MIL-STD-810D, Method 516.3, Procedure VI.
4.6.2.2 Procedure
Step 1: Place each piece of equipment on a level floor or table, as
for normal operation or servicing.
Step 2: Make provision, if necessary, to restrain lateral movement
of the equipment or its supports at one edge of the device. Vertical
rotation about that edge shall not be restrained.
Step 3: Using that edge as a pivot, raise the opposite edge to an
angle of 45 degrees, to a height of four inches above the surface, or
until the point of balance has been reached, whichever occurs first.
Step 4: Release the elevated edge so that it may drop to the test
surface without restraint.
Step 5: Repeat steps 3 and 4 for a total of six events.
Step 6: Repeat steps 2, 3, and 4 for the other base edges, for a
total of 24 drops for each device.
4.6.3 Vibration Test
The vibration test simulates stresses faced during transport of
voting machines and ballot counters between storage locations and
polling places.
4.6.3.1 Applicability
All systems and components, regardless of type, shall meet the
requirements of this test. This test is equivalent to the procedure of
MIL-STD-810D, Method 514.3, Category 1--Basic Transportation, Common
Carrier.
4.6.3.2 Procedure
Step 1: Install the test item in its transit or combination case as
prepared for transport.
Step 2: Attach instrumentation as required to measure the applied
excitation.
Step 3: Mount the equipment on a vibration table with the axis of
excitation along the vertical axis of the equipment.
Step 4: Apply excitation as shown in MIL-STD-810D, Method 514.3-1,
``Basic transportation, common carrier, vertical axis'', with low
frequency excitation cutoff at 10 Hz, for a period of 30 minutes.
Step 5: Repeat steps 2 and 3 for the transverse and longitudinal
axes of the equipment with the excitation profiles shown in Figures
514.3-2 and 514.3-3, respectively. (Note: The total excitation period
equals 90 minutes, with 30 minutes excitation along each axis.)
Step 6: Remove the test item from its transit or combination case
and verify its continued operability.
4.6.4 Low Temperature Test
The low temperature test simulates stresses faced during storage of
voting machines and ballot counters.
4.6.4.1 Applicability
All systems and components, regardless of type, shall meet the
requirements of this test. This test is equivalent to the procedure of
MIL-STD-810D, Method 502.2, Procedure I-Storage. The minimum
temperature shall be -4 degrees F.
4.6.4.2 Procedure
Step 1: Arrange the equipment as for storage. Install it in the
test chamber.
Step 2: Lower the internal temperature of the chamber at any
convenient rate, but not so rapidly as to cause condensation in the
chamber, and in any case no more rapidly than 10 degrees F per minute,
until an internal temperature of -4 degrees F has been reached.
Step 3: Allow the chamber temperature to stabilize. Maintain this

[[Page 19033]]

temperature for a period of 4 hours after stabilization.
Step 4: Allow the internal temperature of the chamber to return to
standard laboratory conditions, at a rate not exceeding 10 degrees F
per minute Step 5: Allow the internal temperature of the equipment to
stabilize at laboratory conditions before removing it from the chamber.
Step 6: Remove the equipment from the chamber and from its
containers, and inspect the equipment for evidence of damage.
Step 7: Verify continued operability of the equipment.
4.6.5 High Temperature Test
The high temperature test simulates stresses faced during storage
of voting machines and ballot counters.
4.6.5.1 Applicability
All systems and components, regardless of type, shall meet the
requirements of this test. This test is equivalent to the procedure of
MIL-STD-810D, Method 501.2, Procedure I-Storage. The maximum
temperature shall be 140 degrees F.
4.6.5.2 Procedure
Step 1: Arrange the equipment as for storage. Install it in the
test chamber.
Step 2: Raise the internal temperature of the chamber at any
convenient rate, but in any case no more rapidly than 10 degrees F per
minute, until an internal temperature of 140 degrees F has been
reached.
Step 3: Allow the chamber temperature to stabilize. Maintain this
temperature for a period of 4 hours after stabilization.
Step 4: Allow the internal temperature of the chamber to return to
standard laboratory conditions, at a rate not exceeding 10 degrees F
per minute.
Step 5: Allow the internal temperature of the equipment to
stabilize at laboratory conditions before removing it from the chamber.
Step 6: Remove the equipment from the chamber and from its
containers, and inspect the equipment for evidence of damage.
Step 7: Verify continued operability of the equipment.
4.6.6 Humidity Test
The humidity test simulates stresses faced during storage of voting
machines and ballot counters.
4.6.6.1 Applicability
All systems and components regardless of type shall meet the
requirements of this test. This test is similar to the procedure of
MIL-STD-810D, Method 507.2, Procedure I-Natural Hot-Humid. It is
intended to evaluate the ability of the equipment to survive exposure
to an uncontrolled temperature and humidity environment during storage.
This test lasts for ten days.
4.6.6.2 Procedure
Step 1: Arrange the equipment as for storage. Install it in the
test chamber.
Step 2 Adjust the chamber conditions to those given in MIL-STD-810D
Table 507.2-I, for the time 0000 of the HotHumid cycle (Cycle 1).
Step 3: Perform a 24-hour cycle with the time and temperature-
humidity values specified in Figure 507.2-1, Cycle 1.
Step 4: Repeat Step 2 until 5, 24-hour cycles have been completed.
Step 5: Continue with the test commencing with the conditions
specified for time = 0000 hours.
Step 6: At any convenient time in the interval between time = 120
hours and time = 124 hours, place the equipment in an operational
configuration, and perform a complete operational status check as
defined in Subsection 4.6.1.5
Step 7: If the equipment satisfactorily completes the status check,
continue with the sixth 24-hour cycle.
Step 8: Perform 4 additional 24-hour cycles, terminating the test
at time = 240 hours
Step 9: Remove the equipment from the test chamber and inspect it
for any evidence of damage.
Step 10: Verify continued operability of the equipment.

4.7 Environmental Tests, Operating

This section addresses a range of tests for all voting system
equipment, including equipment for both precinct count and central
count systems.
4.7.1 Temperature and Power Variation Tests
This test is similar to the low temperature and high temperature
tests of MIL-STD810D, Method 502.2 and Method 501.2, with test
conditions that correspond to the requirements of the performance
standards. This procedure tests system operation under various
environmental conditions for at least 163 hours. During 48 hours of
this operating time, the device shall be in a test chamber. For the
remaining hours, the equipment shall be operated at room temperature.
The system shall be powered for the entire period of this test; the
power may be disconnected only if necessary for removal of the system
from the test chamber.
Operation shall consist of ballot-counting cycles, which vary with
system type. An output report need not be generated after each counting
cycle; the interval between reports, however, should be no more than 4
hours to keep to a practical minimum the time between the occurrence of
a failure or data error and its detection.

Test Ballots per Counting Cycle

Precinct count systems--100 ballots/hour
Central count systems--300 ballots/hour

The recommended pattern of votes is one chosen to facilitate visual
recognition of the reported totals; this pattern shall exercise all
possible voting locations. System features such as data quality tests,
error logging, and audit reports shall be enabled during the test.
Each operating cycle shall consist of processing the number of
ballots indicated in the preceding chart.
Step 1: Arrange the equipment in the test chamber. Connect as
required and provide for power, control and data service through
enclosure wall.
Step 2: Set the supply voltage at 117 vac.
Step 3: Power the equipment, and perform an operational status
check as in Section 4.6.1.5.
Step 4: Set the chamber temperature to 50 degrees F observing
precautions against thermal shock and condensation.
Step 5: Begin 24 hour cycle.
Step 6: At T=4 hrs, lower the supply voltage to 105 vac.
Step 7: At T=8 hrs, raise the supply voltage to 129 vac.
Step 8: At T=11:30 hrs, return the supply voltage to 117 vac and
return the chamber temperature to lab ambient, observing precautions
against thermal shock and condensation.
Step 9: At T=12:00 hrs, raise the chamber temperature to 95 degrees
Fahrenheit.
Step 10: Repeat Steps 5 through 8, with temperature at 95 degrees
Fahrenheit, complete at T=24 hrs.
Step 11: Set the chamber temperature at 50 degrees Fahrenheit as in
Step 4.
Step 12: Repeat the 24 hour cycle as in Steps 5-10, complete at
T=48 hrs.
Step 13: After completing the second 24 hour cycle, disconnect
power from the system and remove it from the chamber if needed.
Step 14: Reconnect the system as in Step 2, and continue testing
for the remaining period of operating time required until the ACCEPT/
REJECT criteria of Subsection 4.7.11 have been met.
4.7.1.1 Data Accuracy
As indicated in Volume I, Section 3, data accuracy is defined in
terms of

[[Page 19034]]

ballot position error rate. This rate applies to the voting functions
and supporting equipment that capture, record, store, consolidate and
report the specific selections, and absence of selections, made by the
voter for each ballot position. Volume I, Section 3.2.1 identifies the
specific functions to be tested.
For each processing function, the system shall achieve a target
error rate of no more than one in 10,000,000 ballot positions, with a
maximum acceptable error rate in the test process of one in 500,000
ballot positions. This error rate includes errors from any source while
testing a specific processing function and it related equipment.
This error rate is used to determine the vote position processing
volume used to test system accuracy for each function:
If the system makes one error before counting 26,997
consecutive ballot positions correctly, it will be rejected. The vendor
is then required to improve the system.
If the system reads at least 1,549,703 consecutive ballot
positions correctly, it will be accepted.
If the system correctly reads more than 26,997 ballot
positions but less than 1,549,703 when the first error occurs, the
testing will have to be continued until another 1,576,701 consecutive
ballot positions are counted without error (a total of 3,126,404 with
one error).
Volume II, Appendix C, Section C.5 provides further details of the
calculation for this testing volume.
4.7.2 Maintainability Test
The ITA shall test for maintainability based on the provisions of
Volume I, Section 3 for maintainability, including both physical
attributes and additional attributes regarding the ease of performing
maintenance activities. These tests include:
a. Examine the physical attributes of the system to determine
whether significant impediments exist for the performance of those
maintenance activities that are to be performed by the jurisdiction.
These activities shall be identified by the vendor in the system
maintenance procedures (part of the TDP).
b. Performing activities designated as maintenance activities for
the jurisdiction in the TDP, in accordance with the instructions
provided by the vendor in the system maintenance procedures, noting any
difficulties encountered.
Should significant impediments or difficulties be encountered that
are not remedied by the vendor, the ITA shall include such findings in
the qualification test results of the qualification test report.
4.7.3 Reliability Test
The ITA shall test for reliability based on the provisions of
Volume I, Section 3 for the acceptable mean time between failure
(MBTF). The MBTF shall be measured during the conduct of other system
performance tests specified in this section, and shall be at least 163
hours. Volume II, Appendix C, Section C.4 provides further details of
the calculation for this testing period.
4.7.4 Availability Test
The ITA shall assess the adequacy of system availability based on
the provisions of Volume I, Section 3. As described in this section,
availability of voting system equipment is determined as a function of
reliability, and the mean time to repair the system in the event of
failure.
Availability cannot be tested directly before the voting system is
deployed in jurisdictions, but can be modeled mathematically to predict
availability for a defined system configuration. This model shall be
prepared by the vendor, and shall be validated by the ITA.
The model shall reflect the equipment used for a typical system
configuration to perform the following system functions:
a. For all paper-based systems:
(1) Recording voter selections (such as by ballot marking or
punch);
(2) Scanning the punches or marks on paper ballots and converting
them into digital data;
b. For all DRE systems:
(1) Recording and storing the voter's ballot selections.
c. For precinct-count systems (paper-based and DRE):
(1) Consolidation of vote selection data from multiple precinct-
based systems to generate jurisdiction-wide vote counts, including
storage and reporting of the consolidated vote data; and
d. For central-count systems (paper-based and DRE):
(1) Consolidation of vote selection data from multiple counting
devices to generate jurisdiction-wide vote counts, including storage
and reporting of the consolidated vote data.
The model shall demonstrate the predicted availability of the
equipment that supports each function. This demonstration shall reflect
the equipment reliability, mean time to repair and assumptions
concerning equipment availability and deployment of maintenance
personnel stated by the vendor in the TDP.

4.8 Other Environmental Tests

4.8.1 Power Disturbance
The test for power disturbance disruption shall be conducted in
compliance with the test specified in in IEC 61000-4-11 (1994-06).
4.8.2 Electromagnetic Radiation
The test for electromagnetic radiation shall be conducted in
compliance with the FCC Part 15 Class B requirements by testing per
ANSI C63.4.
4.8.3 Electrostatic Disruption
The test for electrostatic disruption shall be conducted in
compliance with the test specified in IEC 61000-4-2 (1995-01).
4.8.4 Electromagnetic Susceptibility
The test for electromagnetic susceptibility shall be conducted in
compliance with the test specified in IEC 61000-4-3 (1996).
4.8.5 Electrical Fast Transient
The test for electrical fast transient protection shall be
conducted in compliance with the test specified in IEC 61000-4-4 (1995-
01).
4.8.6 Lightning Surge
The test for lightning surge protection shall be conducted in
compliance with the test specified in IEC 61000-4-5 (1995-02).
4.8.7 Conducted RF Immunity
The test for conducted RF immunity shall be conducted in compliance
with the test specified in IEC 61000-4-6 (1996-04).
4.8.8 Magnetic Fields Immunity
The test for AC magnetic fields RF immunity shall be conducted in
compliance with the test specified in IEC 61000-4-8 (1993-06).

Volume II, Section 5

Table of Contents

5 Software Testing

5.1 Scope
5.2 Basis of Software Testing
5.3 Initial Review of Documentation
5.4 Source Code Review
5.4.1 Control Constructs
5.4.1.1 Replacement Rule
5.4.1.2 Figures
5.4.2 Assessment of Coding Conventions

5 Software Testing

5.1 Scope

This section contains a description of the testing to be performed
by the ITA to confirm the proper functioning of the software components
of a voting system submitted for qualification testing. It

[[Page 19035]]

describes the scope and basis for software testing, the initial review
of documentation to support software testing, and the review of the
voting system source code. Further testing of the voting system
software is addressed in the following sections:
a. Volume II, Section 3, for specific tests of voting system
functionality; and
b. Volume II, Section 6, for testing voting system security and for
testing the operation of the voting system software together with other
voting system components.

5.2 Basis of Software Testing

ITAs shall design and perform procedures that test the voting
system software requirements identified in Volume I. All software
components designed or modified for election use shall be tested in
accordance with the applicable procedures contained in this section.
Unmodified, general purpose COTS non-voting software (e.g.,
operating systems, programming language compilers, data base management
systems, and Web browsers) is not subject to the detailed examinations
specified in this section. However, the ITA shall examine such software
to confirm the specific version of software being used against the
design specification to confirm that the software has not been
modified. Portions of COTS software that have been modified by the
vendor in any manner are subject to review.
Unmodified COTS software is not subject to code examination.
However, source code generated by a COTS package and embedded in
software modules for compilation or interpretation shall be provided in
human readable form to the ITA.
The ITA may inspect COTS source code units to determine testing
requirements or to verify the code is unmodified.
The ITA may inspect the COTS generated software source code in
preparation of test plans and to provide some minimal scanning or
sampling to check for embedded code or unauthorized changes. Otherwise,
the COTS source code is not subject to the full code review and
testing. For purposes of code analysis, the COTS units shall be treated
as unexpanded macros.
Compatibility of the voting system software components or
subsystems with one another, and with other components of the voting
system environment, shall be determined through functional tests
integrating the voting system software with the remainder of the
system.
The specific procedures to be used shall be identified in the
Qualification Test Plan prepared by the ITA. These procedures may
replicate testing performed by the vendor and documented in the
vendor's TDP, but shall not rely on vendor testing as a substitute for
software testing performed by the ITA.
Recognizing variations in system design and the technologies
employed by different vendors, the ITAs shall design test procedures
that account for these variations.

5.3 Initial Review of Documentation

Prior to initiating the software review, the ITA shall verify that
the documentation submitted by the vendor in the TDP is sufficient to
enable:
a. Review of the source code; and
b. Design and conducting of tests at every level of the software
structure to verify that the software meets the vendor's design
specifications and the requirements of the performance standards.

5.4 Source Code Review

The ITA shall compare the source code to the vendor's software
design documentation to ascertain how completely the software conforms
to the vendor's specifications. Source code inspection shall also
assess the extent to which the code adheres to the requirements in
Volume I, Section 4.
5.4.1 Control Constructs
Voting system software shall use the control constructs identified
in this section as follows:
a. If the programming language used does not provide these control
constructs, the vendor shall provide them (that is, comparable control
structure logic). The constructs shall be used consistently throughout
the code. No other constructs shall be used to control program logic
and execution;
b. While some programming languages do not create programs as
linear processes, stepping from an initial condition, through changes,
to a conclusion, the program components nonetheless contain procedures
(such as ``methods'' in object-oriented languages). Even in these
programming languages, the procedures must execute through these
control constructs (or their equivalents, as defined and provided by
the vendor); and
c. Operator intervention or logic that evaluates received or stored
data shall not re-direct program control within a program routine.
Program control may be re-directed within a routine by calling
subroutines, procedures, and functions, and by interrupt service
routines and exception handlers (due to abnormal error conditions). Do-
While (False) constructs and intentional exceptions (used as GoTos) are
prohibited.
Illustrations of control construct techniques are provided in
Figures 4-1 through 4-6.
Fig. 4-1 Sequence
Fig. 4-2 If-Then-Else
Fig. 4-3 Do-While
Fig. 4-4 Do-Until
Fig. 4-5 Case
Fig. 4-6 General loop, including the special case FOR loop
5.4.1.1 Replacement Rule
In the constructs shown, any `process' may be replaced by a simple
statement, a subroutine or function call, or any of the control
constructs. In Fig 4-1 for example, ``Process A'' may be a simple
statement and ``Process B'' another Sequence construct.
BILLING CODE 6820-KF-P

[[Page 19036]]

[GRAPHIC] [TIFF OMITTED] TN12AP06.013

[[Page 19037]]

[GRAPHIC] [TIFF OMITTED] TN12AP06.014

[[Page 19038]]

[GRAPHIC] [TIFF OMITTED] TN12AP06.015

[[Page 19039]]

[GRAPHIC] [TIFF OMITTED] TN12AP06.016

The counter is initialized to zero, if the counter test is false,
the DO process is executed and the counter is incremented (or
decremented). Once the counter test is true, control exits from the
loop without incrementing the counter. The implementation of the FOR
loop in many languages, however, can be error prone. The use of the FOR
loop shall include strictly enforced coding conventions to avoid the
common errors such as a loop that never ends.
The GENERAL LOOP should not be used where one of the other loop
structures will serve. It too is error prone and may not be supported
in many languages without using GOTOs type redirections. However, if
defined in the language, it may be useful in defining some loops where
the exit needs to occur in the middle. Also, in other languages the
GENERAL LOOP logic can be used to simulate the other control
constructs. Like the special case, the use of the GENERAL LOOP shall
require the strict enforcement of coding conventions to avoid problems.
5.4.2 Assessment of Coding Conventions
The ITA shall test for compliance with the coding conventions
specified by the vendor. If the vendor does not identify an appropriate
set of coding conventions in accordance with the provisions of Volume
I, section 4.2.6.a, the ITA shall review the code to ensure that it:
a. Uses uniform calling sequences. All parameters shall either be
validated for type and range on entry into each unit or the unit
comments shall explicitly identify the type and range for the reference
of the programmer and tester. Validation may be performed implicitly by
the compiler or explicitly by the programmer;

[[Page 19040]]

b. For C based language and others to which this applies, has the
return explicitly defined for callable units such as functions or
procedures (do not drop through by default) and, in the case of
functions, have the return value explicitly assigned. Where the return
is only expected to return a successful value, the C convention of
returning zero shall be used or the use of another code justified in
the comments. If an uncorrected error occurs so the unit must return
without correctly completing its objective, a non-zero return value
shall be given even if there is no expectation of testing the return.
An exception may be made where the return value of the function has a
data range including zero;
c. Does not use macros that contain returns or pass control beyond
the next statement;
d. For those languages with unbound arrays, provides controls to
prevent writing beyond the array, string, or buffer boundaries;
e. For those languages with pointers or which provide for
specifying absolute memory locations, provides controls that prevent
the pointer or address from being used to overwrite executable
instructions or to access inappropriate areas where vote counts or
audit records are stored;
f. For those languages supporting case statements, has a default
choice explicitly defined to catch values not included in the case
list;
g. Provides controls to prevent any vote counter from overflowing.
Assuming the counter size is large enough such that the value will
never be reached is not adequate;
h. Is indented consistently and clearly to indicate logical levels;
i. Excluding code generated by commercial code generators, is
written in small and easily identifiable modules, with no more than 50%
of all modules exceeding 60 lines in length, no more than 5% of all
modules exceeding 120 lines in length, and no modules exceeding 240
lines in length. ``Lines'' in this context, are defined as executable
statements or flow control statements with suitable formatting and
comments. The reviewer should consider the use of formatting, such as
blocking into readable units, which supports the intent of this
requirement where the module itself exceeds the limits. The vendor
shall justify any module lengths exceeding this standard;
j. Where code generators are used, the source file segments
provided by the code generators should be marked as such with comments
defining the logic invoked and, if possible, a copy of the source code
provided to the ITA with the generated source code replaced with an
unexpanded macro call or its equivalent;
k. Has no line of code exceeding 80 columns in width (including
comments and tab expansions) without justification;
l. Contains no more than one executable statement and no more than
one flow control statement for each line of source code;
m. In languages where embedded executable statements are permitted
in conditional expressions, the single embedded statement may be
considered a part of the conditional expression. Any additional
executable statements should be split out to other lines;
n. Avoids mixed-mode operations. If mixed mode usage is necessary,
then all uses shall be identified and clearly explained by comments;
o. Upon exit() at any point, presents a message to the user
indicating the reason for the exit().
p. Uses separate and consistent formats to distinguish between
normal status and error or exception messages. All messages shall be
self-explanatory and shall not require the operator to perform any
look-up to interpret them, except for error messages that require
resolution by a trained technician.
q. References variables by fewer than five levels of indirection
(i.e. a.b.c.d or a[b].c->d).
r. Has functions with fewer than six levels of indented scope,
counted as follows:

int function()

[[Page 19041]]

[GRAPHIC] [TIFF OMITTED] TN12AP06.017

u. Has all constants other than 0 and 1 defined or enumerated, or
shall have a comment which clearly explains what each constant means in
the context of its use. Where ``0'' and ``1'' have multiple meanings in
the code unit, even they should be identified. Example: ``0'' may be
used as FALSE, initializing a counter to zero, or as a special flag in
a non-binary category.
v. Only contains the minimum implementation of the ``a = b ? c :
d'' syntax. Expansions such as ``j=a?(b?c:d):e;'' are prohibited.
w. Has all assert() statements coded such that they are absent from
a production compilation. Such coding may be implemented by ifdef()s
that remove them from or include them in the compilation. If
implemented, the initial program identification in setup should
identify that assert() is enable and active as a test version.

Volume II, Section 6

Table of Contents

6 System Level Integration Testing

6.1 Scope
6.2 Basis of Integration Testing
6.2.1 Testing Breadth
6.2.2 System Baseline for Testing
6.2.3 Testing Volume
6.3 Testing Interfaces of System Components
6.4 Security Testing
6.4.1 Access Control
6.4.2 Data Interception and Disruption
6.5 Accessibility Testing
6.6 Physical Configuration Audit
6.7 Functional Configuration Audit

6 System Level Integration Testing

6.1 Scope

This section contains a description of the testing to be performed
by the ITAs to confirm the proper functioning of the fully integrated
components of a voting system submitted for qualification testing. It
describes the scope and basis

[[Page 19042]]

for integration testing, testing of internal and external system
interfaces, testing of security capabilities, and the configuration
audits, including the testing of system documentation.
System-level qualification tests address the integrated operation
of both hardware and software, along with any telecommunications
capabilities. The system-level qualification tests shall include the
tests (functionality, volume, stress, usability, security, performance,
and recovery) indicated in the ITAs' Qualification Test Plan, described
in Appendix A. These tests assess the system's response to a range of
both normal and abnormal conditions initiated in an attempt to
compromise the system. These tests may be part of the audit of the
system's functional attributes, or may be conducted separately.
The system integration tests include two audits: A Physical
Configuration Audit that focuses on physical attributes of the system,
and a Functional Configuration Audit that focuses on the system's
functional attributes, including attributes that go beyond the specific
requirements of the Standards.

6.2 Basis of Integration Testing

This subsection addresses the basis for integration testing, the
system baseline for testing, and data volumes for testing.
6.2.1 Testing Breadth
ITAs shall design and perform procedures that test the voting
system capabilities for the system as a whole. These procedures follow
the testing of the systems hardware and software, and address voting
system requirements defined in Volume I, Sections 2, 5, 6 and 8.
These procedures shall also address the requirements for testing
system functionality provided in Volume II, Section 3. Where practical,
the ITA will perform coverage reporting of the software branches
executed in the functional testing. The selection of the baseline test
cases will follow an operational profile of the common procedures,
sequencing, and options among the shared state requirements and those
that are specifically recognized and supported by the vendor. The ITA
will use the coverage report to identify any portions of the source
code that were not covered and determine:
a. The additional functional tests that are needed;
b. Where more detailed source code review is needed; or
c. Both of the above.
The specific procedures to be used shall be identified in the
Qualification Test Plan prepared by the ITA. These procedures may
replicate testing performed by the vendor and documented in the
vendor's TDP, but shall not rely on vendor testing as a substitute for
testing performed by the ITA.
Recognizing variations in system design and the technologies
employed by different vendors, the ITAs shall design test procedures
that account for these variations.
6.2.2 System Baseline for Testing
The system level qualification tests are conducted using the
version of the system as it is intended to be sold by the vendor and
delivered to jurisdictions. To ensure that the system version tested is
the correct version, the ITA shall witness the build of the executable
version of the system immediately prior to or as part of the physical
configuration audit. Additionally, should components of the system be
modified or replaced during the qualification testing process, the ITA
shall require the vendor conduct a new ``build'' of the system to
ensure that the qualified executable release of the system is built
from tested components.
6.2.3 Testing Volume
For all systems, the total number of ballots to be processed by
each precinct counting device during these tests shall reflect the
maximum number of active voting positions and the maximum number of
ballot styles that the TDP claims the system can support.

6.3 Testing Interfaces of System Components

The ITA shall design and perform test procedures that test the
interfaces of all system modules and subsystems with each other against
the vendor's specifications. These tests shall be documented in the
ITA's Qualification Test Plan, and shall include the full range of
system functionality provided by the vendor's specifications, including
functionality that exceeds the specific requirements of the Standards.
Some voting systems may use components or subsystems from
previously tested and qualified systems, such as ballot preparation.
For these scenarios, the ITA shall, at a minimum,
a. Confirm that the version of previously approved components and
subsystems are unchanged; and
b. Test all interfaces between previously approved modules/
subsystems and all other system modules and subsystems. Where a
component is expected to interface with several different products,
especially from different manufacturers, the vendor shall provide a
public data specification of files or data objects used to exchange
information.
Some systems use telecommunications capabilities as defined in
Section 5. For those systems that do use such capabilities, components
that are located at the poll site or separate vote counting site shall
be tested for effective interface, accurate vote transmission, failure
detection, and failure recovery. For voting systems that use
telecommunications lines or networks that are not under the control of
the vendor (e.g., public telephone networks), the ITA shall test the
interface of vendor-supplied components with these external components
for effective interface, vote transmission, failure detection, and
failure recovery.

6.4 Security Testing

The ITA shall design and perform test procedures that test the
security capabilities of the voting system against the requirements
defined in Volume I, Section 6. These procedures shall focus on the
ability of the system to detect, prevent, log, and recover from a broad
range of security risks as identified in Section 6 and system
capabilities and safeguards, claimed by the vendor in its TDP that go
beyond the risks and threats identified in Volume I, Section 6.
The range of risks tested is determined by the design of the system
and potential exposure to risk. Regardless of system design and risk
profile, all systems are tested for effective access control and
physical data security.
For systems that use public telecommunications networks, including
the Internet, to transmit election management data or official election
results (such as ballots or tabulated results), the ITAs shall conduct
tests to ensure that the system provides the necessary identity-
proofing, confidentiality, and integrity of transmitted data. These
tests shall be designed to confirm that the system is capable of
detecting, logging, preventing, and recovering from types of attacks
known at the time the system is submitted for qualification.
The ITA may meet these testing requirements by confirming proper
implementation of proven commercial security software. In this case,
the vendor must provide the published standards and methods used by the
U.S. Government to test and accept this software, or it may provide
references to free, publicly available publications of

[[Page 19043]]

these standards and methods, such as government web sites.
At its discretion, the ITA may conduct or simulate attacks on the
system to confirm the effectiveness of the system's security
capabilities, employing test procedures approved by the NASED Voting
Systems Board.
6.4.1 Access Control
The ITA shall conduct tests of system capabilities and review the
access control policies and procedures and submitted by the vendor to
identify and verify the access control features implemented as a
function of the system. For those access control features built in as
components of the voting system, the ITA shall design tests to confirm
that these security elements work as specified.
Specific activities to be conducted by the ITA shall include:
a. A review of the vendor's access control policies, procedures and
system capabilities to confirm that all requirements of Volume I,
Section 6.2 have been addressed completely; and
b. Specific tests designed by the ITA to verify the correct
operation of all documented access control procedures and capabilities,
including tests designed to circumvent controls provided by the vendor.
These tests shall include:
(1) Performing the activities that the jurisdiction will perform in
specific accordance with the vendor's access control policy and
procedures to create a secure system, including procedures for software
(including firmware) installation (as described in Volume I, Section
6.4); and
(2) Performing tests intended to bypass or otherwise defeat the
resulting security environment. These tests shall include simulation of
attempts to physically destroy components of the voting system in order
to validate the correct operation of system redundancy and backup
capabilities.
This review applies to the full scope of system functionality. It
includes functionality for defining the ballot and other pre-voting
functions, as well as functions for casting and storing votes, vote
canvassing, vote reporting, and maintenance of the system's audit
trail.
6.4.2 Data Interception and Disruption
For systems that use telecommunications to transmit official voting
data, the ITA shall review, and conduct tests of, the data interception
and prevention safeguards specified by the vendor in its TDP. The ITA
shall evaluate safeguards provided by the vendor to ensure their proper
operation, including the proper response to the detection of efforts to
monitor data or otherwise compromise the system.
For systems that use public communications networks the ITA shall
also review the vendor's documented procedures for maintaining
protection against newly discovered external threats to the
telecommunications network. This review shall assess the adequacy of
such procedures in terms of:
a. Identification of new threats and their impact;
b. Development or acquisition of effective countermeasures;
c. System testing to ensure the effectiveness of the
countermeasures;
d. Notification of client jurisdictions that use the system of the
threat and the actions that should be taken;
e. Distribution of new system releases or updates to current system
users; and
f. Confirmation of proper installation of new system releases.

6.5 Accessibility Testing

The ITA shall design and perform procedures that test the
capability of the voting system to assist voters with disabilities. ITA
test procedures shall confirm that:
a. Voting machines intended for use by voters with disabilities
provide the capabilities required by Volume I, Section 2.2.7;
b. Voting machines intended for use by voters with disabilities
operate consistent with vendor specifications and documentation; and
c. Voting machines intended for use by voters with disabilities
meet all other functional requirements required by Volume I, Section 2.

6.6 Physical Configuration Audit

The Physical Configuration Audit compares the voting system
components submitted for qualification to the vendor's technical
documentation, and shall include the following activities:
a. The audit shall establish a configuration baseline of the
software and hardware to be tested. It shall also confirm whether the
vendor's documentation is sufficient for the user to install, validate,
operate, and maintain the voting system. MIL-STD-1521 can be used as a
guide when conducting this audit;
b. The test agency shall examine the vendor's source code against
the submitted documentation during the Physical Configuration Audit to
verify that the software conforms to the vendor's specifications. This
review shall include an inspection of all records of the vendor's
release control system. If changes have been made to the baseline
version, the test agency shall verify that the vendor's engineering and
test data are for the software version submitted for qualification;
c. If the software is to be run on any equipment other than a COTS
mainframe data processing system, minicomputer, or microcomputer, the
Physical Configuration Audit shall also include a review of all
drawings, specifications, technical data, and test data associated with
the system hardware. This examination shall establish the system
hardware baseline associated with the software baseline;
d. To assess the adequacy of user acceptance test procedures and
data, vendor documents containing this information shall be reviewed
against the system's functional specifications. Any discrepancy or
inadequacy in the vendor's plan or data shall be resolved prior to
beginning the system-level functional and performance tests; and
e. All subsequent changes to the baseline software configuration
made during the course of qualification testing shall be subject to
reexamination. All changes to the system hardware that may produce a
change in software operation shall also be subject to reexamination.
The vendor shall provide a list of all documentation and data to be
audited, cross-referenced to the contents of the TDP. Vendor technical
personnel shall be available to assist in the performance of the
Physical Configuration Audit.

6.7 Functional Configuration Audit

The Functional Configuration Audit encompasses an examination of
vendor tests, and the conduct of additional tests, to verify that the
system hardware and software perform all the functions described in the
vendor's documentation submitted for the TDP. It includes a test of
system operations in the sequence in which they would normally be
performed, and shall include the following activities (MIL-STD-1521 may
be used as a guide when conducting this audit.):
a. The test agency shall review the vendor's test procedures and
test results to determine if the vendor's specified functional
requirements have been adequately tested. This examination shall
include an assessment of the adequacy of the vendor's test cases and
input data to exercise all system functions, and to detect program
logic and data processing errors, if such be present; and
b. The test agency shall perform or supervise the performance of
additional tests to verify nominal system performance in all operating
modes, and to verify on a sampling basis the

[[Page 19044]]

vendor's test data reports. If vendor developmental test data is
incomplete, the ITA shall design and conduct all appropriate module and
integrated functional tests. The functional configuration audit may be
performed in the facility either of the test agency or of the vendor,
and shall use and verify the accuracy and completeness of the System
Operations, Maintenance, and Diagnostic Testing Manuals.
The vendor shall provide a list of all documentation and data to be
audited, cross-referenced to the contents of the TDP. Vendor technical
personnel shall be available to assist in the performance of the
Functional Configuration Audit.

Volume II, Section 7

Table of Contents

7 Examination of Vendor Practices for Configuration Management and
Quality Assurance

7.1 Scope
7.2 Basis of Examinations
7.3 General Examinations Sequence
7.3.1 Examination of Vendor Practices in Parallel with Other
Qualification Testing
7.3.2 Performance of Functional Configuration Audit as an
Element of Integrated System Testing
7.4 Examination of Configuration Management Practices
7.4.1 Configuration Management Policy
7.4.2 Configuration Identification
7.4.3 Baseline, Promotion, and Demotion Procedures
7.4.4 Configuration Control Procedures
7.4.5 Release Process
7.4.6 Configuration Audits
7.4.7 Configuration Management Resources
7.5 Examination of Quality Assurance Practices
7.5.1 Quality Assurance Policy
7.5.2 Parts & Materials Special Tests and Examinations
7.5.3 Quality Conformance Inspections
7.5.4 Documentation

7 Examination of Vendor Practices for Configuration Management and
Quality Assurance

7.1 Scope

This section contains a description of the examination performed by
the ITAs to confirm conformance with the requirements for configuration
management and quality assurance of voting systems. It describes the
scope and basis for the examinations, the general sequence of the
examinations within the overall test process, and provides guidance on
the substantive focus of the examinations.

7.2 Basis of Examinations

ITAs shall design and perform procedures that examine documented
vendor practices for quality assurance and configuration management as
addressed by Volume I, Sections 7 and 8, and complemented by Volume II,
Section 2.
Examination procedures shall be designed and performed by the ITA
that address:
a. Conformance with the requirements to provide information on
vendor practices required by the Standards;
b. Conformance of system documentation and other information
provided by the vendor with the documented practices for quality
assurance and configuration management.
The Standards do not require on-site examination of the vendor's
quality assurance and configuration management practices during the
system development process. However, the ITAs conduct several
activities while at the vendor site to witness the system build that
enable assessment of the vendor's quality assurance and configuration
management practices and conformance with them. These include surveys,
interviews with individuals at all levels of the development team, and
examination of selected internal work products such as system change
requests and problem tracking logs.
It is recognized that examinations of vendor practices, and
determinations of conformance, entail a significant degree of
professional judgement. These standards for vendor practices identify
specific areas of focus for the ITAs, while at the same time relying on
their expertise and professional judgement, as evaluated in the
certification of the ITAs.
The specific procedures used by the ITA shall be identified in the
Qualification Test Plan. Recognizing variations in vendors' quality
assurance and configuration management practices and procedures, the
ITAs shall design examination procedures that account for these
variations.

7.3 General Examinations Sequence

There is no required sequence for performing the examinations of
quality assurance and configuration management practices. No other
testing within the overall qualification testing process is dependent
on the performance and results of these examinations. However,
examinations pertaining to configuration management, in particular
those pertaining to configuration identification, will generally be
useful in understanding the conventions used to define and document the
components of the system and will assist other elements of the
qualification test process.
7.3.1 Examination of Vendor Practices in Parallel With Other
Qualification Testing
While not required, ITAs are encouraged to initiate the
examinations of quality assurance and configuration management
practices early in the overall qualification testing sequence, and
conduct them in parallel with other testing of the voting system.
Conducting these examinations in parallel is recommended to minimize
the overall duration of the qualification process,
7.3.2 Performance of Functional Configuration Audit as an Element of
Integrated System Testing
As described in Volume I, Section 8, the functional configuration
audit verifies that the voting system performs all the functions
described in the system documentation.
To help ensure an efficient test process, this audit shall be
conducted by ITAs as an element of integrated system testing that
confirms the proper functioning of the system as a whole. Integrated
system testing is described in more detail in Volume II, Section 6.

7.4 Examination of Configuration Management Practices

The examination of configuration management practices shall address
the full scope of requirements described in Volume I, Section 8, and
the documentation requirements described in Volume II, Section 2. In
addition to confirming that all required information has been
submitted, the ITAs shall determine the vendor's conformance with the
documented configuration management practices.
7.4.1 Configuration Management Policy
The ITAs shall examine the vendor's documented configuration
management policy to confirm that it:
a. Addresses the full scope of the system, including components
provided by external suppliers; and
b. Addresses the full breadth of system documentation;
7.4.2 Configuration Identification
The ITAs shall examine the vendor's documented configuration
identification practices policy to confirm that they:
a. Describe clearly the basis for classifying configuration items
into categories and subcategories, for numbering of configuration
items; and for naming of configuration items; and
b. Describe clearly the conventions used to identify the version of
the system as a whole and the versions of any lower level elements
(e.g., subsystems, individual elements) if

[[Page 19045]]

such lower level version designations are used.
7.4.3 Baseline, Promotion, and Demotion Procedures
The ITA shall examine the vendor's documented baseline, promotion
and demotion procedures to confirm that they:
a. Provide a clear, controlled process that promotes components to
baseline status when specific criteria defined by the vendor are met;
and
b. Provide a clear controlled process for demoting a component from
baseline status when specific criteria defined by the vendor are met;
7.4.4 Configuration Control Procedures
The ITA shall examine the vendor's configuration control procedures
to confirm that they:
a. Are capable of providing effective control of internally
developed system components; and
b. Are capable of providing effective control of components
developed or supplied by third parties.
7.4.5 Release Process
The ITA shall examine the vendor's release process to confirm that
it:
a. Provides clear accountability for moving forward with the
release of the initial system version and subsequent releases;
b. Provides the means for clear identification of the system
version being replaced;
c. Confirms that all required internal vendor tests and audits
prior to release have been completed successfully;
d. Confirms that each system version released to customers has been
qualified by a the appropriate ITA prior to release;
e. Confirms that each system release has been received by the
customer; and
f. Confirms that each system release has been installed
successfully by the customer;
7.4.6 Configuration Audits
The ITA shall examine the vendor's configuration audit procedures
to confirm that they:
a. Are sufficiently broad in scope to address the entire system,
including system documentation;
b. Are conducted with appropriate timing to enable effective
control of system versions; and
c. Are sufficiently rigorous to confirm that all system
documentation prepared and maintained by the vendor indeed matches the
actual system functionality, design, operation and maintenance
requirements.
7.4.7 Configuration Management Resources
The ITA shall examine the configuration management resource
information submitted by the vendor to determine whether sufficient
information has been provided to enable another organization to clearly
identify the resources used and acquire them for use. This examination
is intended to ensure that in the event the vendor concludes business
operations, sufficient information has been provided to enable an in-
depth audit of the system should such an audit be required by election
officials and/or a law enforcement organization.

7.5 Examination of Quality Assurance Practices

The examination of quality assurance practices shall address the
full scope of requirements described in Volume I, Section 7, and the
documentation requirements described in Volume II, Section 2. The ITA
shall confirm that all required information has been submitted, and
assess whether the vendor's quality assurance program provides for:
a. Clearly measurable quality standards;
b. An effective testing program throughout the system development
life cycle;
c. Application of the quality assurance program to external
providers of system components and supplies;
d. Comprehensive monitoring of system performance in the field and
diagnosis of system failures;
e. Effective record keeping of system failures to support analysis
of failure patterns and potential causes; and
f. Effective processes for notifying customers of system failures
and corrective measures that need to be taken, and for confirming that
such measures are taken.
In addition to the general examinations described above, the ITA
shall focus on the specific elements of the vendor's quality assurance
program indicated below.
7.5.1 Quality Assurance Policy
The ITA shall examine the vendor's quality assurance policy to
confirm that it:
a. Addresses the full scope of the voting system;
b. Clearly designates a senior level individual accountable for
implementation and oversight of quality assurance activities;
c. Clearly designates the individuals, by position within the
vendor's organization, who are to conduct each quality assurance
activity; and
d. Provides procedures that determine compliance with, and correct
deviations from, the quality assurance program at a minimum annually.
7.5.2 Parts & Materials Special Tests and Examinations
The ITA shall examine the vendor's parts and materials special
tests and examinations to confirm that they:
a. Identify appropriate criteria that are used to determine the
specific system components for which special tests are required to
confirm their suitability for use in a voting system;
b. Are designed in a manner appropriate to determine suitability;
and
c. Have been conducted and documented for all applicable parts and
materials.
7.5.3 Quality Conformance Inspections
The ITAs shall examine the vendor's quality conformance plans,
procedures and inspection results to confirm that:
a. All components have been tested according to the test
requirements defined by the vendor;
b. All components have passed the requisite tests; and
c. For each test, the test documentation identifies:
(1) Test location;
(2) Test date;
(3) Individual who conducted the test; and
(4) Test outcome.
7.5.4 Documentation
The ITAs shall examine the vendor's voting system documentation to
confirm that it meets the content requirements of Volume I, Section
7.5, and Volume I Section 2, and is written in a manner suitable for
use by purchasing jurisdictions.

Volume II, Appendix A

Table of Contents

A Qualification Test Plan A-1

A.1 Scope
A.1.1 References
A.1.2 Terms and Abbreviations
A.2 Prequalification Tests
A.3 Materials Required for Testing
A.3.1 Software
A.3.2 Equipment
A.3.3 Test Materials
A.3.4 Deliverable Materials
A.3.5 Proprietary Data
A.4 Test Specifications
A.4.1 Hardware Configuration and Design
A.4.2 Software System Functions
A.4.3 Test Case Design
A.4.3.1 Hardware Qualitative Examination Design
A.4.3.2 Hardware Environmental Test Case Design
A.4.3.3 Software Module Test Case Design and Data

[[Page 19046]]

A.4.3.4 Software Functional Test Case Design
A.4.3.5 System-level Test Case Design
A.5 Test Data
A.5.1 Data Recording
A.5.2 Test Data Criteria
A.5.3 Test Data Reduction
A.6 Test Procedure and Conditions
A.6.1 Facility Requirements
A.6.2 Test Set-up
A.6.3 Test Sequence
A.6.4 Test Operations Procedures

A Qualification Test Plan

A.1 Scope

This Appendix contains a recommended outline for the Qualification
Test Plan, which is to be prepared by the test agency. The primary
purpose of the test plan is to document the test agency's development
of the complete or partial qualification test. A sample outline of a
Qualification Test Plan is illustrated in Figure A-1 at the end of this
Appendix.
It is intended that the test agency use this Appendix as a guide in
preparing a detailed test plan, and that the scope and detail of the
requirements for qualification be tailored to the type of hardware, and
the design and complexity of the software being tested. Required
hardware tests are defined in Section 4, whereas software and system-
level tests must be developed based on the vendor prequalification
tests and information available on the specific software's physical and
functional configuration.
Prior to development of any test plan, the test agency must obtain
the Technical Data Package (TDP) from the vendor submitting the voting
system for qualification. The TDP contains information necessary to the
development of a Qualification Test Plan, such as the vendor's Hardware
Specifications, Software Specifications, System Operating Manual and
System Maintenance Manual.
It is foreseen that vendors may submit some voting systems in use
at the time the standards are issued to partial qualification tests. It
is also specified by the standards that voting systems incorporating
the vendor's software and COTS hardware need only be submitted for
software and system-level tests. Requalification of systems with
modified software or hardware is also anticipated. The test agency
shall alter the test plan outline as required by these situations.
The following sections describe the individual sections of the
recommended Qualification Test Plan.
The test agency shall include the identification, and a brief
description of, the hardware and software to be tested, and any special
considerations that affect the test design and procedure.
A.1.1 References
The test agency shall list all documents that contain material used
in preparing the test plan. This list shall include specific reference
to applicable portions of the standards, and to the vendor's TDP.
A.1.2 Terms and Abbreviations
The test agency shall list and define all terms and phrases
relevant to the hardware, the software, or the test plan.

A.2 Prequalification Tests

The test agency shall evaluate vendor tests, or other agency tests
in determining the scope of testing required for system qualification.
Prequalification test activities may be particularly useful in
designing software functional test cases and tests of system security.
The ITA shall summarize prequalification test results that support
the discussion of the preceding section.

A.3 Materials Required for Testing

The following materials must presented to the ITA in order to
facilitate testing of the voting system:
Software;
Equipment;
Test materials;
Deliverable materials; and
Proprietary Data.
A.3.1 Software
The ITA shall list all software required for the performance of
hardware, software, telecommunications, security and integrated system
tests. If the test environment requires supporting software such as
operating systems, compilers, assemblers, or database managers, then
this software shall also be listed.
A.3.2 Equipment
The ITA shall list all equipment required for the performance of
the hardware, software, telecommunications, security and integrated
system tests. This list shall include system hardware, general purpose
data processing and communications equipment, and test instrumentation,
as required.
A.3.3 Test Materials
The ITA shall list all test materials required in the performance
of the test including, as applicable, test ballot layout and generation
materials, test ballot sheets, test ballot cards and control cards,
standard and optional output data report formats, and any other
materials used to simulate preparation for and conduct of elections.
A.3.4 Deliverable Materials
The ITA shall list all documents and materials to be delivered as a
part of the system, such as:
Hardware specification;
Software specification;
Voter, operator, and hardware and software maintenance
manuals;
Program listings, facsimile ballots, tapes; and
Sample output report formats.
A.3.5 Proprietary Data
The ITA shall list and describe all documentation and data that are
the private property of the vendor, and hence are subject to
restrictions with respect to ITA use, release, or disclosure.

A.4 Test Specifications

The ITA shall cite the pertinent hardware qualitative examinations
and quantitative tests that follow from Volume I, Sections 3 and 9. The
ITA shall also describe the specific test requirements that follow from
the design of the software and telecommunications capabilities under
test.
The qualification test shall include ITA consideration of hardware,
software and telecommunications, design; and ITA development and
conduct of all tests to demonstrate satisfactory performance.
Environmental, non-operating tests shall be performed in the categories
of simulated environmental conditions specified by the vendor or user
requesting the tests. Environmental operating tests shall be performed
under varying temperatures. Other functional tests shall be conducted
in an environment that simulates, as nearly as possible, the intended
use environment.
Test hardware and software shall be identical to that designed to
be used together in the voting system, except that software intended
for use with general-purpose off-the-shelf hardware may be tested using
any equivalent equipment capable of supporting its operation and
functions.
A.4.1 Hardware Configuration and Design
The ITA shall document the hardware configuration and design in
detail sufficient to identify the specific equipment being tested. This
document shall provide a basis for the specific test design and include
a brief description of the intended use of the hardware.

[[Page 19047]]

A.4.2 Software System Functions
The ITA shall describe the software functions in sufficient detail
to provide a foundation for selecting the test case designs and
conditions contained in Subsections A.4.4.3, A.4.4.4, and A.4.4.5,
below. On the basis of this test case design, the ITA shall prepare a
table delineating software functions and how each shall be tested.

A.4.3 Test Case Design

The ITA shall examine the test case design of the following aspects
of the voting system:
Hardware Qualitative Examination Design;
Hardware Environmental Test Case Design;
Software Module Test Case Design and Data;
Software Functional Test Case Design; and
System-level Test Case Design.
A.4.3.1 Hardware Qualitative Examination Design
The ITA shall review the results, submitted by the vendor, of any
previous examinations of the equipment to be tested. The results of
these examinations shall be compared to the performance characteristics
specified by Section 2 of the standards concerning the requirements
for:
Overall system capabilities;
Pre-voting functions;
Voting functions; and
Post-voting functions.
In the event that a review of the results of previous examinations
indicates problem areas, the test agency shall provide a description of
further examinations required prior to conducting the environmental and
system-level tests. If no previous examinations have been performed, or
records of these tests are not available, the test agency shall specify
the appropriate tests to be used in the examination.
A.4.3.2 Hardware Environmental Test Case Design
The ITA shall review the documentation, submitted by the vendor, of
the results and design of any previous environmental tests of the
equipment submitted for testing. The test design and results shall be
compared to the qualification tests described in Volume I, Section 9 of
the standards. The test agency shall cite any additional tests
required, based on this review and those tests requested by the vendor
or the state. The test agency shall also cite any environmental tests
of Section 9 that are not to be conducted, and note the reasons why.
For complete qualification, environmental tests shall include the
following tests, depending upon the design and intended use of the
hardware.
a. Non-operating tests, including the:
(1) Bench handling test;
(2) Vibration test;
(3) Low temperature test;
(4) High temperature test; and
(5) Humidity test; and
b. Operating tests involving a series of procedures that test
system reliability and accuracy under various temperatures and voltages
relevant to election use.
A.4.3.3 Software Module Test Case Design and Data
The test agency shall review the vendor's program analysis,
documentation, and, if available, module test case design. The test
agency shall evaluate the test cases for each module, with respect to
flow control parameters and data on both entry and exit. All
discrepancies between the Software Specifications and the test case
design shall be corrected by the vendor prior to initiation of the
qualification test.
If the vendor's module test case design does not provide conclusive
coverage of all program paths, then the test agency shall perform an
independent analysis to assess the frequency and consequence of error
of the untested paths. The ITA shall design additional module test
cases, as required, to provide coverage of all modules containing
untested paths with potential for untrapped errors.
The test agency shall also review the vendor's module test data in
order to verify that the requirements of the Software Specifications
have been demonstrated by the data.
In the event that the vendor's module test data are insufficient,
the test agency shall provide a description of additional module tests,
prerequisite to the initiation of functional tests.
A.4.3.4 Software Functional Test Case Design
The test agency shall review the vendor's test plans and data to
verify that the individual performance requirements described in Volume
II, Section 2, Subsection 2.5.3.5, are reflected in the software.
As a part of this process, the test agency shall review the
vendor's functional test case designs. The test agency shall prepare a
detailed matrix of system functions and the test cases that exercise
them. The test agency shall also prepare a test procedure describing
all test ballots, operator procedures, and the data content of output
reports. Abnormal input data and operator actions shall be defined.
Test cases shall also be designed to verify that the system is able to
handle and recover from these abnormal conditions.
The vendor's test case design may be evaluated by any standard or
special method appropriate; however, emphasis shall be placed on those
functions where the vendor data on module development reflects
significant debugging problems, and on functional tests that resulted
in disproportionately high error rates.
The test agency shall define ACCEPT/REJECT criteria for
qualification using the Software Specifications and, if the software
runs on special hardware, the associated Hardware Specifications to
determine acceptable ranges of performance.
The test agency shall describe the functional tests to be
performed. Depending upon the design and intended use of the voting
system, all or part of the functions listed below shall be tested.
a. Ballot preparation subsystem;
b. Test operations performed prior to, during, and after processing
of ballots, including:
(1) Logic tests to verify interpretation of ballot styles, and
recognition of precincts to be processed;
(2) Accuracy tests to verify ballot reading accuracy;
(3) Status tests to verify equipment statement and memory contents;
(4) Report generation to produce test output data; and
(5) Report generation to produce audit data records;
c. Procedures applicable to equipment used in the polling place
for:
(1) Opening the polling place and enabling the acceptance of
ballots; (b) maintaining a count of processed ballots;
(2) Monitoring equipment status;
(3) Verifying equipment response to operator input commands;
(4) Generating real-time audit messages;
(5) Closing the polling place and disabling the acceptance of
ballots;
(6) Generating election data reports;
(7) Transfer of ballot counting equipment, or a detachable memory
module, to a central counting location; and
(8) Electronic transmission of election data to a central counting
location; and
d. Procedures applicable to equipment used in a central counting
place:
(1) Initiating the processing of a ballot deck or PMD for one or
more precincts;
(2) Monitoring equipment status;

[[Page 19048]]

(3) Verifying equipment response to operator input commands;
(4) Verifying interaction with peripheral equipment, or other data
processing systems;
(5) Generating real-time audit messages;
(6) Generating precinct-level election data reports;
(7) Generating summary election data reports;
(8) Transfer of a detachable memory module to other processing
equipment;
(9) Electronic transmission of data to other processing equipment;
and
(10) Producing output data for interrogation by external display
devices.
A.4.3.5 System-Level Test Case Design
The test agency shall provide a description of system tests of both
the software and hardware. For software, these tests shall be designed
according the stated design objective without consideration of its
functional specification. The test agency shall independently prepare
the system test cases to assess the response of the hardware and
software to a range of conditions, such as:
Volume tests: These tests investigate the system's
response to processing more than the expected number of ballots/voters
per precinct, to processing more than the expected number of precincts,
or to any other similar conditions that tend to overload the system's
capacity to process, store, and report data;
Stress tests: These tests investigate the system's
response to transient overload conditions. Polling place devices shall
be subjected to ballot processing at the high volume rates at which the
equipment can be operated to evaluate software response to hardware-
generated interrupts and wait states. Central counting systems shall be
subjected to similar overloads, including, for systems that support
more than one card reader, continuous processing through all readers
simultaneously;
Usability tests: These tests are designed to exercise
characteristics of the software such as response to input control or
text syntax errors, error message content, audit message content, and
other features contained in the software design objectives but not
directly related to a functional specification;
Accessibility tests: These tests are designed to exercise
system capabilities and features intended for use by voters with
disabilities in accordance with Volume I, Section 2.2.5;
Security tests: These tests are designed to defeat the
security provisions of the system including modification or disruption
of pre-voting, voting, and post voting processing; unauthorized access
to, deletion, or modification of data, including audit trail data; and
modification or elimination of security mechanisms;
Performance tests: These tests verify accuracy, processing
rate, ballot format handling capability, and other performance
attributes claimed by the vendor; and
Recovery tests: These tests verify the ability of the
system to recover from hardware and data errors.

A.5 Test Data

A.5.1 Data Recording
The test agency shall identify all data recording requirements
(e.g.; what is to be measured, how tests and results are to be
recorded). The test agency shall also design or approve the design of
forms or other recording media to be employed. The test agency shall
supply any special instrumentation (pulse measuring device) needed to
satisfy the data requirements.
A.5.2 Test Data Criteria
The test agency shall describe the criteria against which test
results will be evaluated, such as the following:
Tolerances: These criteria define the acceptable range for
system performance. These tolerances shall be derived from the
applicable hardware performance requirements contained in Volume I,
Section 3, Hardware Standards.
Samples: These criteria define the minimum number of
combinations or alternatives of input and output conditions that can be
exercised to constitute an acceptable test of the parameters involved.
Events: These criteria define the maximum number of
interrupts, halts or other system breaks that may occur due to nontest
conditions. This count shall not include events from which recovery
occurs automatically or where a relevant status message is displayed.
A.5.3 Test Data Reduction
The test agency shall describe the techniques to be used for
processing test data. These techniques may include manual, semi-
automatic, or fully automatic reduction procedures. However, semi-
automatic and automatic procedures shall have been shown to be capable
of handling the test data accurately and properly. They shall also
produce an item-by-item comparison of the data and the embedded
acceptance criteria as output.

A.6 Test Procedure and Conditions

The test agency shall describe the test conditions and procedures
for performing the tests. If tests are not to be performed in random
order, this section shall contain the rationale for the required
sequence, and the criteria that must be met, before the sequence can be
continued. This section shall also describe the procedure for setting
up the equipment in which the software will be tested, for system
initialization, and for performing the tests. Each of the following
sections that contain a description of a test procedure shall also
contain a statement of the criteria by which readiness and successful
completion shall be indicated and measured.
A.6.1 Facility Requirements
The test agency shall describe the space, equipment,
instrumentation, utilities, manpower, and other resources required to
support the test program.
A.6.2 Test Set-up
The test agency shall describe the procedure for arranging and
connecting the system hardware with the supporting hardware and
telecommunications equipment, if applicable. It shall also describe the
procedure required to initialize the system, and to verify that it is
ready to be tested.
A.6.3 Test Sequence
The test agency shall state any restrictions on the grouping or
sequence of tests in this section.
A.6.4 Test Operations Procedures
The test agency shall provide the step-by-step procedures for each
test case to be conducted. Each step shall be assigned a test step
number and this number, along with critical test data and test
procedures information, shall be tabulated onto a test report form for
test control and the recording of test results.
In this section, the test agency shall also identify all test
operations personnel, and their respective duties. In the event that
the operator procedure is not defined in the vendor's operations or
user manual, the test agency shall also provide a description of the
procedures to be followed by the test personnel.

Figure A-1

Test Plan Outline

1 Introduction

1.1 References
1.2 Terms and Abbreviations

2 Prequalification Tests

2.1 Prequalification Test Activity

[[Page 19049]]

2.2 Prequalification Test Results

3 Materials Required for Testing

3.1 Software
3.2 Equipment
3.3 Test Materials
3.4 Deliverable Materials
3.5 Proprietary Data

4 Test Specification

4.1 Requirements
4.2 Hardware Configuration and Design
4.3 Software System Functions
4.4 Test Case Design
4.4.1 Hardware Qualitative Examination Design
4.4.2 Hardware Environmental Test Case Design
4.4.3 Software Module Test Case Design and Data
4.4.4 Software Functional Test Case Design and Data
4.4.5 System-level Test Case Design

5 Test Data

5.1 Data Recording
5.2 Test Data Criteria
5.3 Test Data Reduction

6 Test Procedure and Conditions

6.1 Facility Requirements
6.2 Test Set-up
6.3 Test Sequence
6.4 Test Operations Procedures

Volume II, Appendix B

Table of Contents

B Qualification Test Report
B.1 Scope
B.1.1 New Voting System Qualification Test Report
B.1.2 Changes to Previously Qualified Voting System
Qualification Test Report
B.2 Qualification Test Background
B.3 System Identification
B.4 System Overview
B.5 Qualification Test Results and Recommendation
B.6 Appendix--Test Operations and Findings
B.7 Appendix--Test Data Analysis

B Qualification Test Report

B.1 Scope

This Appendix contains a recommended outline for the Qualification
Test Report to be prepared by the test agency. The test report shall be
organized so as to facilitate the presentation of conclusions and
recommendations regarding system acceptability, a summary of the test
operations, a summary of the test results, the test data records, and
the analyses that support the conclusions and recommendations. The
content of the report may vary based on the scope of review conducted.
B.1.1 New Voting System Qualification Test Report
A full report is prepared for the initial qualification testing of
a voting system. This document consists of five main sections:
Introduction, Qualification Test Background, System Identification,
System Overview, and Qualification Test Results.
Detailed information about the test operations and findings, and
test data, are included as appendices to the report.
Sections B.2 through B.8 describe the contents of the individual
sections of this report.
B.1.2 Changes to Previously Qualified Voting System Qualification Test
Report
This report addresses a wide range of scenarios. After a
preliminary review of the submitted changes, the test agency may
determined that:
a. A review of all change documentation against the baseline
materials was sufficient for recommendation for qualification; or
b. All changes must be retested against the previously qualified
baseline; or
c. The scope of the changes are substantial enough such that a
complete retest of the software is required.
The format of this report varies, based on the type of review that
was performed. If only a review of change documentation against the
baseline materials was performed the report is quite simple. It
consists of an Introduction, a Version Description, the Testing
Approach, and a Results Summary. A more extensive report is prepared,
for changes that have extensive impact on the system design and/or
operations.

B.2 Qualification Test Background

This section contains the following information:
a. General information about the qualification test process; and
b. A list and definition of all terms and nomenclature peculiar to
the hardware, the software, or the test report;

B.3 System Identification

This section gives information about the tested software and
supporting hardware, including:
a. System name and major subsystems (or equivalent);
b. System Version;
c. Test Support Hardware; and
d. Specific documentation provided in the vendor's TDP used to
support testing.

B.4 System Overview

This section describes the voting system in terms of its overall
design structure, technologies used, processing capacity claimed by the
vendor for system components (such as ballot counters, voting machines,
vote consolidation equipment) and mode of operation. It may also
identify other products that interface with the voting system.

B.5 Qualification Test Results and Recommendation

This section provides a summary of the results of the testing
process, and indicates any special considerations that affect the
conclusions derived from the test results. This summary includes:
a. The acceptability of the system design and construction based on
the performance of the system hardware, software and communications,
and on the source code inspection;
b. The degree to which the hardware and software meet the vendor's
specifications and the standards, and the acceptability of the vendor's
technical and user documentation;
c. General findings on the maintainability of the system including,
where applicable, notation of specific maintenance activities that are
determined to be difficult to perform;
d. Identification and description of any deficiencies that remain
uncorrected after completion of the qualification test and that has
caused or is judged to be capable of causing the loss or corruption of
voting data, providing sufficient detail to support a recommendation to
reject the system being tested. (Similarly, any deficiency in
compliance with the security, accuracy, data retention, and audit
requirements are fully described); and
e. A specific recommendation to the NASED ITA Committee for
approval or rejection.
Of note, any uncorrected deficiency that does not involve the loss
or corruption of voting data shall not necessarily be cause for
rejection. Deficiencies of this type may include failure to fully
achieve the levels of performance specified in Volume I, Sections 3 and
4 of the Standards, or failure to fully implement formal programs for
quality assurance and configuration management described in Volume I,
Sections 7 and 8. The nature of the deficiency is described in detail
sufficient to support the recommendation either to accept or to reject
the system, and the recommendation is based on consideration of the
probable effect the deficiency will have on safe and efficient system
operation during all phases of election use.

[[Page 19050]]

B.6 Appendix--Test Operations and Findings

This appendix provides additional detail about the test results to
enable the understanding of test results and recommendation. This
information is organized in a manner that reflects the Qualification
Test Plan. Summaries of the results of hardware examinations, operating
and non-operating hardware tests, software module tests, software
function tests, and system-level tests (including security and
telecommunications tests, and the results of the Physical and
Functional Configuration Audits) are provided.

B.7 Appendix--Test Data Analysis

This appendix provides summary records of the test data and the
details of the analysis. The analysis includes a comparison of the
vendor's hardware and software specifications to the test data,
together with any mathematical or statistical procedure used for data
reduction and processing.

Volume II, Appendix C

Table of Contents

C Appendix C: Qualification Test Design Criteria

C.1 Scope
C.2 Approach to Test Design
C.3 Probability Ratio Sequential Test (PRST)
C.4 Time-Based Failure Testing Criteria
C.5 Accuracy Testing Criteria

C Appendix C: Qualification Test Design Criteria

C.1 Scope

This appendix describes the guiding principles used to design the
voting system qualification testing process conducted by ITAs.
Qualification tests are designed to demonstrate that the system
meets or exceeds the requirements of the Standards. The tests are also
used to demonstrate compliance with other levels of performance claimed
by the manufacturer.
Qualification tests must satisfy two separate and possibly
conflicting sets of considerations. The first is the need to produce
enough test data to provide confidence in the validity of the test and
its apparent outcome. The second is the need to achieve a meaningful
test at a reasonable cost, and cost varies with the difficulty of
simulating expected real-world operating conditions and with test
duration. It is the test designer's job to achieve an acceptable
balance of these constraints.
The rationale and statistical methods of the test designs contained
in the Standards are discussed below. Technical descriptions of their
design can be found in any of several books on testing and statistical
analysis.

C.2 Approach To Test Design

The qualification tests specified in the Standards are primarily
concerned with assessing the magnitude of random errors. They are also,
however, capable of detecting bias errors that would result in the
rejection of the system.
Test data typically produce two results. The first is an estimate
of the true value of some system attribute such as speed, error rate,
etc. The second is the degree of certainty that the estimate is a
correct one. The estimate of an attribute's value may or may not be
greatly affected by the duration of the test. Test duration, however,
is very important to the degree of certainty; as the length of the test
increases, the level of uncertainty decreases. An efficient test design
will produce enough data over a sufficient period of time to enable an
estimate at the desired level of confidence.
There are several ways to design tests. One approach involves the
preselection of some test parameter, such as the number of failures or
other detectable factor. The essential element of this type of design
is that the number of observations is independent of their results. The
test may be designed to terminate after 1,000 hours or 10 days, or when
5 failures have been observed. The number of failures is important
because the confidence interval (uncertainty band) decreases rapidly as
the number of failures increases. However, if the system is highly
reliable or very accurate, the length of time required to produce a
predetermined number of failures or errors using this method may be
unachievably long.
Another approach is to determine that the actual value of some
attribute need not be learned by testing, provided that the value can
be shown to be better than some level. The test would not be designed
to produce an estimate of the true value of the attribute but instead
to show, for example, that reliability is at least 123 hours or the
error rate is no greater than one in ten million characters.
The latter design approach, which was chosen for the Standards,
uses what is called Sequential Analysis. Instead of the test duration
being fixed, it varies depending on the outcome of a series of
observations. The test is terminated as soon as a statistically valid
decision can be reached that the factor being tested is at least as
good as or no worse than the predetermined target value. A sequential
analysis test design called the ``Wald Probability Ratio Test'' is used
for reliability and accuracy testing.

C.3 Probability Ratio Sequential Test (PRST)

The design of a Probability Ratio Sequential Test (PRST) requires
that four parameters be specified:

H0, the null hypothesis
H1, the alternate hypothesis

a, the Producer's risk
b, the Consumer's risk

The Standards anticipate using the PRST for testing both time-based
and event-based failures.
This test design provides decision criteria for accepting or
rejecting one of two test hypotheses: the null hypothesis, which is the
Nominal Specification Value (NSV), or the alternate hypothesis, which
is the MAV. The MAV could be either the Minimum Acceptable Value or the
Maximum Acceptable Value depending upon what is being tested.
(Performance may be specified by means of a single value or by two
values. When a single value is specified, it shall be interpreted as an
upper or lower single-sided 90 percent confidence limit. If two values,
these shall be interpreted as a two-sided 90 percent confidence
interval, consisting of the NSV and MAV.)
In the case of Mean Time Between Failure (MTBF), for example, the
null hypothesis is that the true MTBF is at least as great as the
desired value (NSV), while the alternate hypothesis is that the true
value of the MTBF is less than some lower value (Minimum Acceptable
Value). In the case of error rate, the null hypothesis is that the true
error rate is less than some very small desired value (NSV), while the
alternate hypothesis is that the true error rate is greater than some
larger value that is the upper limit for acceptable error (Maximum
Acceptable Value).

C.4 Time-based Failure Testing Criteria

An equivalence between a number of events and a time period can be
established when the operating scenarios of a system can be determined
with precision. Some of the performance test criteria of Volume II,
Section 4, Hardware Testing, use this equivalence.
System acceptance or rejection can be determined by observing the
number of relevant failures that occur during equipment operation. The
probability ratio for this test is derived from the Exponential
probability distribution. This distribution implies a constant hazard
rate for equipment failure that is not dependent on the time of testing
or

[[Page 19051]]

the previous failures. In that case, two or more systems may be tested
simultaneously to accumulate the required number of test hours, and the
validity of the data is not affected by the number of operating hours
on a particular unit of equipment. However, for environmental operating
hardware tests, no unit shall be subjected to less than two complete
24-hour test cycles in a test chamber as required by Volume II,
Subsection 4.7.1 of the Standards.
In this case, the null hypothesis is that the Mean Time Between
Failure (MTBF), as defined in Volume I, Subsection 3.4.3 of the
Standards, is at least as great as some value, here the Nominal
Specification Value. The alternate hypothesis is that the MTBF is no
better than some value, here the Minimum Acceptable Value.
For example, a typical system operations scenario for environmental
operating hardware tests will consist of approximately 45 hours of
equipment operation. Broken down, this time allotment involves 30 hours
of equipment setup and readiness testing and 15 hours of elections
operations. If the Minimum Acceptable Value is defined as 45 hours, and
a test discrimination ratio of 3 is used (in order to produce an
acceptably short expected time of decision), then the Nominal
Specification Value equals 135 hours.
With a value of decision risk equal to 10 percent, there is no more
than a 10 percent chance that a system would be rejected when, in fact,
with a true MTBF of at least 135 hours, the system would be acceptable.
It also means that there is no more than a 10 percent chance that a
system would be accepted with a true MTBF lower than 45 hours when it
should have been rejected.
Therefore,

H0: MTBF = 135 hours
H1: MTBF = 45 hours

a = 0.10
b = 0.10.

Under this PRST design, the test is terminated and an ACCEPT
decision is reached when the cumulative number of equipment hours in
the second column of the following table has been reached, and the
number of failures is equal to or less than the number shown in the
first column. The test is terminated and a REJECT decision is reached
when the number of failures occurs in less than the number of hours
specified in the third column. Here, the minimum time to accept (on
zero failures) is 169 hours. In the event that no decision has been
reached by the times shown in the last table entries, the test is
terminated, and the decision is declared as indicated. Any time that 7
or more failures occur, the test is terminated and the equipment
rejected. If after 466 hours of operation the cumulative failure score
is less than 7.0, then the equipment is accepted.

------------------------------------------------------------------------
Accept if time Reject if time less
Number of failures greater than than
------------------------------------------------------------------------
0................................. 169 Continue test
1................................. 243 Continue test
2................................. 317 26
3................................. 392 100
4................................. 466 175
5................................. 466 249
6................................. 466 323
7................................. N/A (\1\)
------------------------------------------------------------------------
\1\ Terminate and REJECT

This test is based on the table of test times of the truncated PRST
design V-D in the Military Handbook MIL-HDBK-781A that is designated
for discrimination ratio 3 and a nominal value of 0.10 for both a and
b. The Handbook states that the true producer risk is 0.111 and the
true consumer risk is 0.109. Using the theoretical formulas for either
the untruncated or Truncated truncated tests will lead to different
numbers.
The test design will change if given a different set of parameters.
Some jurisdictions may find the Minimum Acceptable Value of 45 hours
unacceptable for their needs. In addition, it may be appropriate to use
a different discrimination ratio, or different Consumer's and
Producer's risk. Also, before using tests based on the MTBF, it should
be determined whether time-based testing is appropriate rather than
event-based or another form of testing. If MTBF-based procedures are
chosen, then the appropriateness of the assumption of a constant hazard
rate with exponential failures should in turn be assessed.

C.5 Accuracy Testing Criteria

Some voting system performance attributes are tested by inducing an
event or series of events, and the relative or absolute time intervals
between repetitions of the event has no significance. Although an
equivalence between a number of events and a time period can be
established when the operating scenarios of a system can be determined
with precision, another type of test is required when such equivalence
cannot be established. It uses event-based failure frequencies to
arrive at ACCEPT/REJECT criteria. This test may be performed
simultaneously with time-based tests.
For example, the failure of a device is usually dependent on the
processing volume that it is required to perform. The elapsed time over
which a certain number of actuation cycles occur is, under most
circumstances, not important. Another example of such an attribute is
the frequency of errors in reading, recording, and processing vote
data.
The error frequency, called ``ballot position error rate,'' applies
to such functions as process of detecting the presence or absence of a
voting punch or mark, or to the closure of a switch corresponding to
the selection of a candidate.
Qualification and acceptance test procedures that accommodate
event-based failures are, therefore, based on a discrete, rather than a
continuous probability distribution. A Probability Ratio Sequential
Test using the binomial distribution is recommended. In the case of
ballot position error rate, the calculation for a specific device (and
the processing function that relies on that device) is based on:

HO: Desired error rate = 1 in 10,000,000
H1: Maximum acceptable error rate = 1 in 500,000

a = 0.05
b = 0.05

and the minimum error-free sample size to accept for qualification
tests is 1,549,703 votes.
The nature of the problem may be illustrated by the following
example, using the criteria contained in the Standards for system error
rate. A target for the desired accuracy is established at

[[Page 19052]]

a very low error rate. A threshold for the worst error rate that can be
accepted is then fixed at a somewhat higher error rate. Next, the
decision risk is chosen, that is, the risk that the test results may
not be a true indicator of either the system's acceptability or
unacceptability. The process is as follows:
The desired accuracy of the voting system, whatever its
true error rate (which may be far better), is established as no more
than one error in every ten million characters (including the null
character).
If it can be shown that the system's true error rate does
not exceed one in every five hundred thousand votes counted, it will be
considered acceptable. (This is more than accurate enough to declare
the winner correctly in almost every election.)
A decision risk of 5 percent is chosen, to be 95 percent
sure that the test data will not indicate that the system is bad when
it is good or good when it is bad.
This results in the following decision criteria:
If the system makes one error before counting 26,997
consecutive ballot positions correctly, it will be rejected. The vendor
is then required to improve the system;
If the system reads at least 1,549,703 consecutive ballot
positions correctly, it will be accepted; and
If the system correctly reads more than 26,997 ballot
positions but less than 1,549,703 when the first error occurs, the
testing will have to be continued until another 1,576,701 consecutive
ballot positions are counted without error (a total of 3,126,404 with
one error).

[FR Doc. 06-3101 Filed 4-11-06; 8:45 am]

BILLING CODE 6820-KF-C