This is the accessible text file for GAO report number GAO-07-1167T 
entitled 'Elections: Status of GAO's Review of Voting Equipment Used in 
Florida's 13th Congressional District' which was released on August 3, 
2007. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Statement: 

Before the Task Force on Florida-13, Committee on House Administration, 
House of Representatives: 

United States Government Accountability Office: 

GAO: 

For Release on Delivery Expected at 10:00 a.m. EDT: 

Friday, August 3, 2007: 

Elections: 

Status of GAO's Review of Voting Equipment Used in Florida's 13th 
Congressional District: 

Statement of Dr. Nabajyoti Barkakati: 
Senior-Level Technologist: 
Center for Technology and Engineering: 
Applied Research and Methods: 

GAO-07-1167T: 

Chairman Gonzalez, Ms. Lofgren, Mr. McCarthy, 

I am pleased to appear before the Task Force today to update you on the 
progress of our review of voting equipment used in Florida's 13th 
Congressional District, which we are conducting in response to your 
request of May 25, 2007. I want to thank the Task Force for its 
continued support of our efforts. We have accomplished a lot in the 
past few weeks, but we still have several work items to complete before 
we can formally draw any conclusions. 

In November 2006, about 18,000 undervotes were reported in Sarasota 
County in the race for Florida's 13th Congressional District.[Footnote 
1] Following the contesting of the election results in the House of 
Representatives, the Task Force met and unanimously voted to seek GAO's 
assistance in determining whether the voting systems contributed to the 
large undervote in Sarasota County. On June 14, 2007, we met with the 
Task Force and agreed upon an engagement plan, which included the 
following review objectives: (1) What voting systems and equipment were 
used in Sarasota County and what processes governed their use? (2) What 
was the scope of the undervote in Sarasota County in the general 
election? (3) To what extent were tests conducted on the voting systems 
in Sarasota County prior to the general election and what were the 
results of those tests? and (4) Considering the tests that were 
conducted on the voting systems from Sarasota County after the general 
election, are additional tests needed to determine whether the voting 
systems contributed to the undervote? 

To conduct our work, we visited Sarasota County twice, most recently 2 
weeks ago, and we were in Tallahassee last week to meet with the 
Secretary of State and the Division of Elections. While in Tallahassee, 
we were able to execute a nondisclosure agreement that permitted us 
access to items that the State of Florida and the manufacturer of the 
voting system, Election Systems and Software (ES&S), considered 
proprietary, including the proprietary appendixes of the Florida State 
University source code review report; the technical data package, which 
includes items such as the software specification; and the source code 
for the firmware installed in the iVotronic touchscreen voting systems 
used in Sarasota County. We are currently working on a separate 
nondisclosure agreement to access technical and testing information 
from ES&S directly. 

In our meetings with Sarasota County, we learned the entire process of 
configuring the election, running the election, and tallying the 
results, and about the testing the county conducts on the voting 
systems, such as the logic and accuracy testing. In our meetings with 
the Division of Elections, we discussed the conduct of certification 
testing, in particular, the testing conducted on the ES&S system used 
in Sarasota County, and the conduct of the state audit--how decisions 
were made to conduct the audit and the processes used to conduct the 
audit. In addition, we have received and are reviewing and analyzing 
data and documentation received from both sources, as well as the 
submissions from the contestant and the contestee provided by the Task 
Force. 

Summary: 

We have identified the voting systems and equipment used in Sarasota 
County and verified that the systems were approved for use by the 
Florida Division of Elections. We know that nine different ballot 
styles were used on the iVotronic touchscreen voting systems and have 
an understanding of how the ballots were configured and loaded onto the 
machines. Further, it was also explained to us how votes are tallied 
and certified, including the conduct of the machine and manual 
recounts. 

We have been analyzing the detailed ballot results from the election as 
well as the incident and technician logs from Sarasota County to 
identify patterns in the undervote. Specifically, we have examined the 
undervote by machine, precinct, and ballot style. Patterns in the 
undervote could provide us insight on specific conditions that could 
have caused the undervote. However, we have not yet noticed any 
apparent patterns, but we are continuing our analysis. From our 
analysis, we have been able to verify that 1,499 iVotronic voting 
systems recorded votes in the 2006 general election and the vote counts 
for the contestant, contestee, and undervotes match the vote totals for 
election day, early voting, and provisional ballots in the Florida-13 
race. A total of 17,846 undervotes were recorded in the Florida-13 race 
out of the 119,919 ballots cast using the iVotronic voting systems-- 
corresponding to a 14.88 percent undervote rate.[Footnote 2] 

While we have not yet completed our review of all of the testing 
efforts to determine whether they provide reasonable assurance that the 
machines properly reflect in their totals the selections made when the 
ballot is cast, there are some preliminary observations we can make. 

A variety of testing is needed to obtain reasonable assurance that this 
objective is accomplished, including ballot testing, load testing, and 
environmental testing.[Footnote 3] As agreed with you, our efforts will 
review the testing that has already been completed, including tests 
conducted by the State of Florida (certification testing), Sarasota 
County (logic and accuracy testing), and the equipment manufacturer. We 
are also reviewing the tests conducted as a part of the state audit, 
including parallel testing, the examination of Sarasota County's 
election practices, and the Florida State University source code 
review. Once we complete our review of the testing efforts, we will 
identify the potential benefits associated with conducting any 
additional tests--how they will help us understand whether the system 
contributed to the undervote issue--and the resources needed to conduct 
such tests. 

So far, we have focused our efforts on two types of tests--ballot 
testing and load testing. With between 28 and 40 contests on the 
Sarasota County ballots in the 2006 general election, the number of 
possible voting combinations is over 100 trillion. Accordingly, it is 
unrealistic to expect that all possible vote combinations can be 
tested. 

We have also examined how the system allowed voters different ways to 
make a selection in the Florida-13 race and recognized that these 
represented different ways that the voters could indicate their intent 
in the race. By taking into account these variations, our analysis has 
found at least 112 different ways a voter could make his or her 
selection and cast the ballot in the Florida-13 race, assuming that it 
was the only race on the ballot. Specifically, a voter could (1) 
initially select either candidate or neither candidate (i.e. 
undervote), (2) change the vote on the initial screen, and (3) use a 
combination of features to change or verify his or her selection by 
using the page back and review screen options. We found that the 
Florida certification tests and the Sarasota County logic and accuracy 
tests verified 3 ways to select a candidate; and the Florida parallel 
tests verified 10 ways to select a candidate--meaning that of the 112 
ways, 13 have been tested. We have not yet assessed whether this is 
significant. 

A test to determine whether a system can handle the expected volume of 
activity is commonly referred to as load testing. We found that ballots 
used for load testing during the certification testing were machine- 
generated using a testing program built into the iVotronic system, 
i.e., users do not touch the screen to make a selection and cast a 
ballot. Neither the Florida audit nor Sarasota County's logic and 
accuracy testing performed load testing. We have not yet assessed 
whether this is significant. 

We have also been reviewing the Florida State University source code 
review. As we mentioned, we obtained access to the source code last 
week and we were able to verify for ourselves some of the items 
discussed in its report. We have had prior discussions with the leader 
of the Florida State review team and will be continuing our discussions 
with the review team and the manufacturer to ensure our understanding 
of both the findings of their review and the operations of the 
iVotronic system. One of the items noted in the report was that the 
review team did not (1) convert the source code to object code, and (2) 
compare the resulting object code to the object code that was used to 
run the voting machines in Sarasota County.[Footnote 4] We are still 
assessing the significance of this item. 

As a part of our review of the state audit, we examined the selection 
of samples for the parallel testing and the review of the Sarasota 
County election practices. Our preliminary analysis has found that 
these sample sizes are too small to support generalization of the 
results to the overall population. For example, the generalization of 
the results from the use of 10 machines for parallel testing cannot be 
supported because the sample drawn was not random and the sample size 
was too small. Similarly, we have little assurance that the examination 
of 6 machines' firmware is adequate to conclude that the firmware was 
not compromised on any of the machines. Our discussions with Florida 
officials indicate that such limitations resulted from court-imposed 
restrictions on machine access and resource considerations of 
performing the testing. 

It is important to bear in mind that these are just our preliminary 
observations. It is not clear to us yet whether these are items we 
think will need to be tested; but they are items we have noticed while 
we are reviewing the previously completed test activities. As we 
previously discussed, for any testing issues we identify, we plan to 
determine how relevant and significant the issue is and the resources 
needed to conduct such tests. Our identification of resources will 
include test personnel and equipment, the voting systems and equipment 
to be tested, and the time required to conduct such tests. For example, 
as we have discussed, one of the issues we identified in the source 
code review is that the source code was not converted to object code 
and compared to ensure that it represented the code used in Sarasota 
County. Further, our preliminary analysis has shown that we do not have 
reasonable assurance that the firmware was not compromised on any of 
the iVotronic systems used during the election. In order to determine 
whether these issues warrant further testing, we still need to 
determine the potential significance of these issues, as well as 
identify the test personnel and equipment, the voting systems and 
equipment to be tested, and the time required to conduct such tests. To 
identify these resources, it will also be important to determine how 
such tests should be structured and executed. 

Besides conducting such resource analyses, we still have several 
activities to complete with regard to testing. First, we have not yet 
evaluated the testing conducted by the system manufacturer, and second, 
we are still in the process of identifying other appropriate tests that 
could be used to determine whether the voting systems caused the 
undervote (for example, the effects of provisional ballots and 
environmental conditions). 

Mr. Chairman, this completes my prepared statement. I would be happy to 
respond to any questions you or the other members of the Task Force may 
have at this time. 

For further information about this testimony, please contact Keith 
Rhodes, Chief Technologist, at (202) 512-6412 or rhodesk@gao.gov, or 
Naba Barkakati at (202) 512-4499 or barkakatin@gao.gov. 

FOOTNOTES 

[1] Undervotes are votes for fewer choices than permitted. In this 
case, it means ballots that did not record a selection for either 
candidate in the congressional contest. 

[2] Because the absentee ballots were not cast using iVotronic voting 
systems, we did not verify the absentee ballot counts. When absentee 
ballots are included, a total of 142,532 ballots were cast and a total 
of 18,412 undervotes were recorded. 

[3] For the purposes of this review, ballot testing is a subset of the 
functional testing that focuses on the vote selection and casting 
functions. This includes testing the different ways in which a voter 
may make selections on a ballot and then cast a ballot with the 
iVotronic electronic voter interface. For example, the Florida Voting 
Systems Standards require the system to allow the user (1) to make a 
selection for each contest, and (2) to review the selections made and 
make any changes prior to the vote being cast. 

Load testing, for the purposes of this review, is the testing performed 
to provide reasonable assurance that the voting system can properly 
handle the expected volume of voters and ballots that are expected. 
Florida certification tests include a test to verify that a precinct 
count system, such as the iVotronic, can process at least 9,900 
ballots. 

According to the Florida Voting System Standards, environmental tests 
are intended to simulate exposure to shock and vibration associated 
with handling and transportation and to temperature conditions. For 
example, voting systems in Florida are to be able to operate in 
temperature conditions ranging between 40 and 100 degrees Fahrenheit. 

[4] According to the Institute of Electrical and Electronics Engineers, 
source code contains computer instructions and data definitions 
expressed in a form suitable for input to an assembler, compiler, or 
other translator that generates the object code. Object code contains 
the computer instructions and data definitions expressed in a form that 
can be recognized by the processing unit of a computer.


GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: