This is the accessible text file for GAO report number GAO-07-1167T entitled 'Elections: Status of GAO's Review of Voting Equipment Used in Florida's 13th Congressional District' which was released on August 3, 2007. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Statement: Before the Task Force on Florida-13, Committee on House Administration, House of Representatives: United States Government Accountability Office: GAO: For Release on Delivery Expected at 10:00 a.m. EDT: Friday, August 3, 2007: Elections: Status of GAO's Review of Voting Equipment Used in Florida's 13th Congressional District: Statement of Dr. Nabajyoti Barkakati: Senior-Level Technologist: Center for Technology and Engineering: Applied Research and Methods: GAO-07-1167T: Chairman Gonzalez, Ms. Lofgren, Mr. McCarthy, I am pleased to appear before the Task Force today to update you on the progress of our review of voting equipment used in Florida's 13th Congressional District, which we are conducting in response to your request of May 25, 2007. I want to thank the Task Force for its continued support of our efforts. We have accomplished a lot in the past few weeks, but we still have several work items to complete before we can formally draw any conclusions. In November 2006, about 18,000 undervotes were reported in Sarasota County in the race for Florida's 13th Congressional District.[Footnote 1] Following the contesting of the election results in the House of Representatives, the Task Force met and unanimously voted to seek GAO's assistance in determining whether the voting systems contributed to the large undervote in Sarasota County. On June 14, 2007, we met with the Task Force and agreed upon an engagement plan, which included the following review objectives: (1) What voting systems and equipment were used in Sarasota County and what processes governed their use? (2) What was the scope of the undervote in Sarasota County in the general election? (3) To what extent were tests conducted on the voting systems in Sarasota County prior to the general election and what were the results of those tests? and (4) Considering the tests that were conducted on the voting systems from Sarasota County after the general election, are additional tests needed to determine whether the voting systems contributed to the undervote? To conduct our work, we visited Sarasota County twice, most recently 2 weeks ago, and we were in Tallahassee last week to meet with the Secretary of State and the Division of Elections. While in Tallahassee, we were able to execute a nondisclosure agreement that permitted us access to items that the State of Florida and the manufacturer of the voting system, Election Systems and Software (ES&S), considered proprietary, including the proprietary appendixes of the Florida State University source code review report; the technical data package, which includes items such as the software specification; and the source code for the firmware installed in the iVotronic touchscreen voting systems used in Sarasota County. We are currently working on a separate nondisclosure agreement to access technical and testing information from ES&S directly. In our meetings with Sarasota County, we learned the entire process of configuring the election, running the election, and tallying the results, and about the testing the county conducts on the voting systems, such as the logic and accuracy testing. In our meetings with the Division of Elections, we discussed the conduct of certification testing, in particular, the testing conducted on the ES&S system used in Sarasota County, and the conduct of the state audit--how decisions were made to conduct the audit and the processes used to conduct the audit. In addition, we have received and are reviewing and analyzing data and documentation received from both sources, as well as the submissions from the contestant and the contestee provided by the Task Force. Summary: We have identified the voting systems and equipment used in Sarasota County and verified that the systems were approved for use by the Florida Division of Elections. We know that nine different ballot styles were used on the iVotronic touchscreen voting systems and have an understanding of how the ballots were configured and loaded onto the machines. Further, it was also explained to us how votes are tallied and certified, including the conduct of the machine and manual recounts. We have been analyzing the detailed ballot results from the election as well as the incident and technician logs from Sarasota County to identify patterns in the undervote. Specifically, we have examined the undervote by machine, precinct, and ballot style. Patterns in the undervote could provide us insight on specific conditions that could have caused the undervote. However, we have not yet noticed any apparent patterns, but we are continuing our analysis. From our analysis, we have been able to verify that 1,499 iVotronic voting systems recorded votes in the 2006 general election and the vote counts for the contestant, contestee, and undervotes match the vote totals for election day, early voting, and provisional ballots in the Florida-13 race. A total of 17,846 undervotes were recorded in the Florida-13 race out of the 119,919 ballots cast using the iVotronic voting systems-- corresponding to a 14.88 percent undervote rate.[Footnote 2] While we have not yet completed our review of all of the testing efforts to determine whether they provide reasonable assurance that the machines properly reflect in their totals the selections made when the ballot is cast, there are some preliminary observations we can make. A variety of testing is needed to obtain reasonable assurance that this objective is accomplished, including ballot testing, load testing, and environmental testing.[Footnote 3] As agreed with you, our efforts will review the testing that has already been completed, including tests conducted by the State of Florida (certification testing), Sarasota County (logic and accuracy testing), and the equipment manufacturer. We are also reviewing the tests conducted as a part of the state audit, including parallel testing, the examination of Sarasota County's election practices, and the Florida State University source code review. Once we complete our review of the testing efforts, we will identify the potential benefits associated with conducting any additional tests--how they will help us understand whether the system contributed to the undervote issue--and the resources needed to conduct such tests. So far, we have focused our efforts on two types of tests--ballot testing and load testing. With between 28 and 40 contests on the Sarasota County ballots in the 2006 general election, the number of possible voting combinations is over 100 trillion. Accordingly, it is unrealistic to expect that all possible vote combinations can be tested. We have also examined how the system allowed voters different ways to make a selection in the Florida-13 race and recognized that these represented different ways that the voters could indicate their intent in the race. By taking into account these variations, our analysis has found at least 112 different ways a voter could make his or her selection and cast the ballot in the Florida-13 race, assuming that it was the only race on the ballot. Specifically, a voter could (1) initially select either candidate or neither candidate (i.e. undervote), (2) change the vote on the initial screen, and (3) use a combination of features to change or verify his or her selection by using the page back and review screen options. We found that the Florida certification tests and the Sarasota County logic and accuracy tests verified 3 ways to select a candidate; and the Florida parallel tests verified 10 ways to select a candidate--meaning that of the 112 ways, 13 have been tested. We have not yet assessed whether this is significant. A test to determine whether a system can handle the expected volume of activity is commonly referred to as load testing. We found that ballots used for load testing during the certification testing were machine- generated using a testing program built into the iVotronic system, i.e., users do not touch the screen to make a selection and cast a ballot. Neither the Florida audit nor Sarasota County's logic and accuracy testing performed load testing. We have not yet assessed whether this is significant. We have also been reviewing the Florida State University source code review. As we mentioned, we obtained access to the source code last week and we were able to verify for ourselves some of the items discussed in its report. We have had prior discussions with the leader of the Florida State review team and will be continuing our discussions with the review team and the manufacturer to ensure our understanding of both the findings of their review and the operations of the iVotronic system. One of the items noted in the report was that the review team did not (1) convert the source code to object code, and (2) compare the resulting object code to the object code that was used to run the voting machines in Sarasota County.[Footnote 4] We are still assessing the significance of this item. As a part of our review of the state audit, we examined the selection of samples for the parallel testing and the review of the Sarasota County election practices. Our preliminary analysis has found that these sample sizes are too small to support generalization of the results to the overall population. For example, the generalization of the results from the use of 10 machines for parallel testing cannot be supported because the sample drawn was not random and the sample size was too small. Similarly, we have little assurance that the examination of 6 machines' firmware is adequate to conclude that the firmware was not compromised on any of the machines. Our discussions with Florida officials indicate that such limitations resulted from court-imposed restrictions on machine access and resource considerations of performing the testing. It is important to bear in mind that these are just our preliminary observations. It is not clear to us yet whether these are items we think will need to be tested; but they are items we have noticed while we are reviewing the previously completed test activities. As we previously discussed, for any testing issues we identify, we plan to determine how relevant and significant the issue is and the resources needed to conduct such tests. Our identification of resources will include test personnel and equipment, the voting systems and equipment to be tested, and the time required to conduct such tests. For example, as we have discussed, one of the issues we identified in the source code review is that the source code was not converted to object code and compared to ensure that it represented the code used in Sarasota County. Further, our preliminary analysis has shown that we do not have reasonable assurance that the firmware was not compromised on any of the iVotronic systems used during the election. In order to determine whether these issues warrant further testing, we still need to determine the potential significance of these issues, as well as identify the test personnel and equipment, the voting systems and equipment to be tested, and the time required to conduct such tests. To identify these resources, it will also be important to determine how such tests should be structured and executed. Besides conducting such resource analyses, we still have several activities to complete with regard to testing. First, we have not yet evaluated the testing conducted by the system manufacturer, and second, we are still in the process of identifying other appropriate tests that could be used to determine whether the voting systems caused the undervote (for example, the effects of provisional ballots and environmental conditions). Mr. Chairman, this completes my prepared statement. I would be happy to respond to any questions you or the other members of the Task Force may have at this time. For further information about this testimony, please contact Keith Rhodes, Chief Technologist, at (202) 512-6412 or rhodesk@gao.gov, or Naba Barkakati at (202) 512-4499 or barkakatin@gao.gov. FOOTNOTES [1] Undervotes are votes for fewer choices than permitted. In this case, it means ballots that did not record a selection for either candidate in the congressional contest. [2] Because the absentee ballots were not cast using iVotronic voting systems, we did not verify the absentee ballot counts. When absentee ballots are included, a total of 142,532 ballots were cast and a total of 18,412 undervotes were recorded. [3] For the purposes of this review, ballot testing is a subset of the functional testing that focuses on the vote selection and casting functions. This includes testing the different ways in which a voter may make selections on a ballot and then cast a ballot with the iVotronic electronic voter interface. For example, the Florida Voting Systems Standards require the system to allow the user (1) to make a selection for each contest, and (2) to review the selections made and make any changes prior to the vote being cast. Load testing, for the purposes of this review, is the testing performed to provide reasonable assurance that the voting system can properly handle the expected volume of voters and ballots that are expected. Florida certification tests include a test to verify that a precinct count system, such as the iVotronic, can process at least 9,900 ballots. According to the Florida Voting System Standards, environmental tests are intended to simulate exposure to shock and vibration associated with handling and transportation and to temperature conditions. For example, voting systems in Florida are to be able to operate in temperature conditions ranging between 40 and 100 degrees Fahrenheit. [4] According to the Institute of Electrical and Electronics Engineers, source code contains computer instructions and data definitions expressed in a form suitable for input to an assembler, compiler, or other translator that generates the object code. Object code contains the computer instructions and data definitions expressed in a form that can be recognized by the processing unit of a computer. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, D.C. 20548: Public Affairs: Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: