This is the accessible text file for GAO report number GAO-05-759T entitled 'Advanced Technology Program: Inherent Factors in Selection Process Are Likely to Limit Identification of Similar Research' which was released on May 31, 2005. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Testimony: Before the Subcommittee on Federal Financial Management, Government Information, and International Security, Committee on Homeland Security and Government Affairs, U.S. Senate: United States Government Accountability Office: GAO: For Release on Delivery Expected at 2:30 p.m. EDT: Thursday, May 26, 2005: Advanced Technology Program: Inherent Factors in Selection Process Are Likely to Limit Identification of Similar Research: Statement of Robin M. Nazzaro, Director, Natural Resources ad Environment: GAO-05-759T: GAO Highlights: Highlights of GAO-05-759T, a report to the Chairman, Subcommittee on Federal Financial Management, Government Information, and International Security, Committee on Homeland Security and Governmental Affairs, U.S. Senate: Why GAO Did This Study: The Advanced Technology Program (ATP) supports research that accelerates the development of high-risk technologies with the potential for broad-based economic benefits for the nation. Under the program, administrators at the National Institute of Standards and Technology are to ensure that they do not fund research that would be conducted in the same period without ATP funding. Between 1990 and September 2004, ATP funded 768 projects at a cost of about $2.3 billion. There is a continuing debate over whether the private sector has sufficient incentives to undertake research on high-risk, high- payoff emerging technologies without government support, such as ATP. This testimony discusses the results of GAO’s April 2000 report, Advanced Technology Program: Inherent Factors in the Selection Process Could Limit Identification of Similar Research (GAO/RCED-00-114) and provides updated information. GAO determined (1) whether ATP had funded projects with research goals that were similar to projects funded by the private sector and (2) if ATP did, whether its award selection process ensures that such research would not be funded in the future. What GAO Found: The three completed ATP-funded projects GAO reviewed, which were approved for funding in 1990 and 1992, addressed research goals that were similar to those already funded by the private sector. GAO chose these 3 projects from among the first 38 completed projects, each representing a different technology sector: computers, electronics, and biotechnology. These three technology sectors represent 26 of the 38 completed ATP projects, or 68 percent. The projects included an on- line handwriting recognition system, a system to increase the capacity of existing fiber optic cables for the telecommunications industry, and a process for turning collagen into fibers for human prostheses use. In the case of the handwriting recognition project, ATP provided $1.2 million to develop a system to recognize cursive handwriting for pen- based (i.e., without a keyboard) computer input. GAO identified several private firms that were conducting similar research on handwriting recognition at approximately the same time the ATP project was funded. In fact, this line of research began in the late 1950s. In addition, GAO identified multiple patents, as early as 5 years prior to the start of the ATP project, in the field of handwriting recognition. GAO found similar results in the other two projects. Two inherent factors in ATP’s award selection process—the need to guard against conflicts of interest and the need to protect proprietary information—make it unlikely that ATP can avoid funding research already being pursued by the private sector in the same time period. These factors, which have not changed since 1990, make it difficult for ATP project reviewers to identify similar efforts in the private sector. For example, to guard against conflicts of interest, the program uses technical experts who are not directly involved with the proposed research. Their acquaintance with ongoing research is further limited by the private sector’s practice of not disclosing its research efforts or results so as to guard proprietary information. As a result, it may be impossible for the program to ensure that it is consistently not funding existing or planned research that would be conducted in the same time period in the absence of ATP financial assistance. GAO made no recommendations in its April 2000 report. www.gao.gov/cgi-bin/getrpt?GAO-05-759T. To view the full product, including the scope and methodology, click on the link above. For more information, contact Robin M. Nazzaro at (202) 512-3841 or nazzaror@gao.gov. [End of section] Dear Mr. Chairman and Members of the Subcommittee: We are pleased to be here today to discuss our past work,[Footnote 1] as well as to provide some updated information, on the funding that the Advanced Technology Program (ATP) provides for private research. As you know, ATP was established in 1988 to support research that accelerates the development of high-risk technologies with the potential for broad- based economic benefits for the nation.[Footnote 2] Under the provisions establishing ATP, program administrators at the National Institute of Standards and Technology (NIST) are to ensure that they are not funding existing or planned research that would be conducted in the same time period in the absence of ATP financial assistance. Between 1990 and September 2004, ATP funded 768 projects at a cost of about $2.3 billion in federal matching funds. Research can provide both private benefits, which accrue to the owners of the research results, and societal benefits, which accrue to society as a whole. In some instances, the private sector does not fund research that would be beneficial to society because doing so might not provide an adequate return on a firm's investment. To address this situation, the federal government, through tax credits or direct public funding, supports research that has very broad societal benefits, such as basic research and research focused on developing technologies in areas such as public health and nutrition, energy conservation, and environmental protection. However, there is a continuing debate over whether the private sector has sufficient incentives to undertake research on high-risk, high-payoff emerging and enabling technologies without government support, such as ATP. In this context, in our prior work, we determined (1) whether, in the past, ATP had funded projects with research goals that were similar to projects funded by the private sector and (2) if we identified such cases, whether ATP's award selection process ensures that such research would not be funded in the future. To determine whether ATP has funded projects similar to private sector projects, we chose 3 of the first 38 completed projects, each representing a different technology sector: biotechnology; electronics; and information, computers, and communications. These three technology sectors represent 26 of the 38, or 68 percent, of the ATP projects completed by 1999. We reviewed the ATP project files and held discussions with industry and academic experts, technical reviewers, and award recipients to assist in our examination of these projects. We also conducted patent searches on the technical areas associated with each of the three projects. Our objective was not to provide an evaluation of the quality of the research funded by ATP or the private sector, nor the impact these projects may or may not have had on their respective industries. To address the second objective, we reviewed ATP's award selection process. We did not review the overall management of the program. We performed our initial work from October 1999 through April 2000, and developed updated information in May 2005, in accordance with generally accepted government auditing standards. Results in Brief: The three completed ATP-funded projects, which were approved for funding in 1990 and 1992, addressed research goals that were similar to those already funded by the private sector. The projects included an on- line handwriting recognition system, a system to increase the capacity of existing fiber optic cables for the telecommunications industry, and a process for turning collagen into fibers for human prostheses use. In the case of the handwriting recognition project, ATP provided $1.2 million to develop a system to recognize cursive handwriting for pen- based (i.e., without a keyboard) computer input. We identified several private firms that were conducting similar research on handwriting recognition at approximately the same time the ATP project was funded. In fact, this line of research began in the late 1950s. In addition, we identified multiple patents, as early as 5 years prior to the start of the ATP project, in the field of handwriting recognition. We found similar results in the other two projects. Two inherent factors in ATP's award selection process--the need to guard against conflicts of interest and the need to protect proprietary information--make it unlikely that ATP can avoid funding research already being pursued by the private sector in the same time period. These factors, which have not changed since 1990, make it difficult for ATP project reviewers to identify similar efforts in the private sector. For example, to guard against conflicts of interest, the program uses technical experts who are not directly involved with the proposed research. Their acquaintance with on-going research is further limited by the private sector's practice of not disclosing its research efforts or results so as to guard proprietary information. As a result, it may be impossible for the program to ensure that it is consistently not funding existing or planned research that would be conducted in the same time period in the absence of ATP financial assistance. Background: ATP, which began funding projects in fiscal year 1990, was intended to fund high-risk research and development (R&D) projects with broad commercial and societal benefits that would not be undertaken by a single company or group of companies, either because the risk was too high or because the economic benefits of success would not accrue to the investors. ATP is viewed as a mechanism for fostering investment in areas in which societal returns would exceed private returns. ATP has addressed other opportunities to achieve broader societal goals, such as small business participation, as well as the establishment of joint ventures for high-risk technologies that would be difficult for any one company to justify because, for example, the benefits spread across the industry as a whole. Thus, ATP is seen by some as a means of addressing market failure in research areas that would otherwise not be funded, thereby facilitating the economic growth that comes from the commercialization and use of new technologies in the private sector. Advocates of the program believe that the government should serve as a catalyst for companies to cooperate and undertake important new work that would not have been possible in the same time period without federal participation. Critics of the program view ATP as industrial policy, or the means by which government rather than the marketplace picks winners and losers. ATP provides funding through cooperative agreements--a type of financial assistance in which the federal government is substantially involved in project management. ATP offers these agreements through announced annual competitions. It provides multiyear funding to single companies and to industry-led joint ventures. The proposal review and selection process is a multistep process based on NIST regulations. In general, these steps include a preliminary screening, technical and business reviews, semifinalist identification, oral reviews, ranking, and final selection. At the beginning of each round of ATP competitions, NIST establishes Source Evaluation Boards (SEBs) to ensure that all proposals receive careful consideration. Each SEB is comprised of NIST technical experts as well as outside specialists with backgrounds in business and economics. ATP supplements the SEBs with outside technical reviewers, generally federal government experts in the specific industry of the proposal. Independent business experts are also hired on a consulting basis, including high-tech venture capitalists, people who teach strategic business planning, retired corporate executives from large and small high-tech businesses, as well as economists and business development specialists. All SEB members and outside reviewers must sign nondisclosure statements, agree to protect proprietary information, and certify that they have no conflicts of interest. As part of the proposal evaluation process, ATP uses the external reviewers to assess the technical and business merit of the proposed research. Each proposal is sponsored by both technical and business SEB members, whose roles include identifying reviewers, summarizing evaluative comments, and making recommendations to the SEB. The SEB evaluates the proposals, selects the semifinalists, conducts oral interviews with semifinalists, and ranks the semifinalists. A source selecting official makes the final award decisions based on the ranked list of proposals from the SEB. The three projects that we reviewed received funding through the ATP competitions announced in 1990 and 1992. In those years, the selection criteria included scientific and technical merit, potential broad-based benefits, technology transfer benefits, the proposing organization's commitment level and organizational structure, and the qualifications and experience of the proposing organization's staff. Each of the five selection criteria was weighted at 20 percent. Today, these same selection criteria are used but are grouped into two categories, each weighted at 50 percent. The "Scientific and Technical Merit" category addresses a variety of issues related to the technical plan and the relevant experience of the proposing organization. The second category, "Potential for Broad-Based Economic Benefits," addresses the means to achieving an economic benefit and commercialization plans, as well as issues related to the proposer's level of commitment, organizational structure, and management plan. Technical and business reviewers complete documentation, referred to as technical and business evaluation worksheets, that address various aspects of these criteria. Three ATP Projects Addressed Similar Research Goals to Projects in the Private Sector: The three completed projects that we reviewed addressed research goals that were similar to goals the private sector was addressing at about the same time. Each of the three projects was from a different sector of technology--computers, electronics, and biotechnology. The projects include (1) an on-line handwriting recognition system for computer input, (2) a system to increase the capacity of existing fiber optic cables for the telecommunications industry, and (3) a process for turning collagen into fibers for human prostheses use. ATP Project on Handwriting Recognition: Both the ATP project and several private sector projects had a similar research goal of developing an on-line system to recognize natural or cursive handwritten data without the use of a keyboard. This technology would make computers more useful where keyboard use is limited by physical problems or in situations where using a keyboard is not practical. On-line handwriting recognition means that the system recognizes handwritten data while the user writes. The primary technical problem in handwriting recognition is that writing styles vary greatly from person to person, depending upon whether the user is in a hurry, fatigued, or subject to a variety of other factors. While the technology for obtaining recognition of constrained careful writing or block print writing was commercially available, systems for cursive writing recognition were not commercially available because of the greater handwriting variability that was encountered. The ATP project we reviewed sought to develop an on-line natural handwriting recognition system that was user-independent and able to translate natural or cursive handwriting. Communication Intelligence Corporation (CIC) was the award recipient. CIC used its ATP funding of $1.2 million from 1991 to 1993 to build its own algorithms and models for developing its handwriting recognition system.[Footnote 3] During the project, CIC created a database that includes thousands of cursive handwriting samples and developed new recognition algorithms. Some of this technology has been incorporated into a registered software product that has the ability to recognize cursive writing in limited circumstances. According to the experts we interviewed, as well as literature and patent searches, several companies were attempting to achieve a similar goal of handwriting recognition through their research around the same time that the ATP project received funding. Some of the key players in the private sector conducting research on cursive handwriting recognition included Paragraph International (in collaboration with Apple Computer) and Lexicus (which later became a division of Motorola). For example, Apple licensed a cursive handwriting recognition system from a Soviet company, Paragraph International, according to articles published in computer magazines in October 1991. According to these sources, this technology provided Apple with a foundation for recognizing printed, cursive, or block handwritten text. Another indication of research with a similar goal appeared in the October 1990 edition of PC Week, which reported that "handwriting recognition is an emerging technology that promises increased productivity both for current microcomputer owners and for a new breed of users armed with hand-held 'pen-based' computers." Similarly a technical journal article indicated that there was renewed interest in the 1980s in this field of on-line handwriting recognition, from its advent in the 1960s, because of more accurate electronic tablets, more compact and powerful computers, and better recognition algorithms.[Footnote 4] Moreover, according to the U.S. Patent and Trademark Office's (PTO) database, over 450 patents were issued on handwriting recognition software, concepts, and related products from 1985 through 1999,[Footnote 5] indicating that research of a similar goal was being conducted around the time of the ATP project. Given the fact that it can take many years between the time a research project takes place and the time that an outcome is realized, this time period for a patent search allowed us to determine whether there was research ongoing during the time of the ATP project. The dates of the patents actually occurred sometime after the research was conducted. And, as we reported in a prior report,[Footnote 6] the time between the point when a patent application is filed until the date when a patent is issued, or the application is abandoned, ranged from 19.8 months to 21 months, adding additional time to when the research was done. ATP Project on Capacity Expansion of Fiber Optic Cables: Another ATP project we reviewed, which proposed to develop a system to increase the capacity of existing fiber optic cables for the telecommunications industry, also had a similar goal to that of research in the private sector. At the same time, firms in the private sector were attempting to increase the number of light signals that can be transmitted through a single strand of fiber optic cable using a technology called wavelength division multiplexing (WDM).[Footnote 7] In the 1980s, telephone companies laid fiber optic cables across the United States and other countries to create an information system that could carry significantly more data than the copper wires they replaced. Tremendous increases in cable traffic, primarily from the Internet, have crowded these cables. WDM technology was aimed at providing a cost-effective alternative to the expensive option of installing additional fiber optic cables. Accuwave Corporation (Accuwave) was the ATP award recipient. Accuwave used its ATP funding of approximately $2 million from March 1993 through March 1995 to develop a wavelength division multiplexing system that would substantially increase the number of signals that could be transmitted through a single optical fiber strand, using the concept of volume holography. Volume holography uses holograms to direct multiple light signals simultaneously through a single fiber strand. Accuwave was able to make improvements on these issues but not enough to fully develop and market a successful WDM system for the telecommunications market. In 1996, a competitor beat Accuwave to the market. After the completion of the ATP project, Accuwave filed for bankruptcy protection due to its inability to successfully commercialize a wavelength division multiplexing system. Other private firms were involved in research with a similar goal of increasing the capacity of fiber optic cable at about the same time as Accuwave was conducting its research. Conceptual research on such systems dates back to the early 1980s, but development and commercialization did not flourish until the mid-to late-1990s. Bell Labs (now Lucent Technologies), Nortel Networks, and Ciena Corporation, among others, were considered some of the major competitors in the industry. In the early 1990s, these firms were attempting to develop WDM technology using different methods and materials. For example, Ciena Corporation developed a system that incorporated fiber-Bragg gratings, which are filters embedded directly onto fiber optic cable that help to separate multiple light signals through a single fiber strand. We also found an indication of WDM-related research through a review of issued patents. According to PTO's database, over 2,000 patents were issued related to wavelength division multiplexing components, systems, and concepts from 1985 through 1999. The patents issued ranged from 10 patents in 1985 to 493 in 1999. ATP Project on Regenerating Tissues and Organs: Both the ATP project and private sector projects we identified in the tissue engineering field had similar broad research goals of developing biological equivalents for defective tissues and organs utilizing diverse technical approaches. ATP's project proposed procedures for extracting, storing, spinning, and weaving collagen (the main constituent of connective tissue and bones) into fibers suitable for human prostheses that could induce the body's cells to regenerate lost tissue. Tissue Engineering, Inc., received ATP's award of about $2 million for use over the years 1993 through 1996. The company's long- term and yet unrealized goal is to transplant these prostheses into humans, after which the collagen framework, or scaffold, would induce the growth and function of normal body cells within it, eventually remodeling lost human tissue and replacing the scaffold. Within the very innovative field of tissue engineering, however, many competitors were attempting to achieve similar broad research goals. Organogenesis, the Collagen Corporation, Integra LifeSciences, Advanced Tissue Sciences, Genzyme Tissue, Osiris Therapeutics, Matrix Pharmaceuticals, and ReGen Biologics are key players in the market to develop structures that could replace or regenerate cells, tissues, and organs such as skin, teeth, orthopedic structures, cartilage, and valves. A number of these companies have subsequently received ATP awards. In addition, universities and medical schools have researchers investigating the many possibilities to engineer human tissues, and eventually complex organs, such as the liver, pancreas, and heart. According to one expert, there is a great deal of competition within the field of tissue engineering. Although the Tissue Engineering, Inc. research focused on the use of collagen as the basis for these structures, other companies were pursuing a variety of technical approaches for addressing the goal of developing biological equivalents for defective tissues and organs. In addition to research in collagen, other companies and researchers have also been attempting to create human tissues and organs from other biological materials, synthetics, and hybrid products, which are both biologic and synthetic. For example, researchers from the Massachusetts Institute of Technology (MIT) developed an artificial skin product using collagen and a natural polymer. Several companies have since developed comparable products. In 1986, researchers from MIT and a hospital in Massachusetts began inserting cells into scaffolds created of biodegradable polymer. As the cells multiply, tissues form. The magazine BusinessWeek reported this concept as "an elegantly simple concept that underlies most engineered tissue."[Footnote 8] Two competitors, Integra LifeSciences and Organogenesis, reported that they were also doing work on the use of collagen in various applications. Although their technical approaches were different than the ATP project, the broad research goals were similar. In addition to our discussions with experts and literature searches, patent research shows that there was activity related to the field of tissue engineering prior to and during the ATP project. According to a search done on the PTO website, at least 370 patents were issued related to cell culturing, scaffolding or matrix development, and tissue engineering from 1985 through 1999. Experts have also indicated that there are several patents related to the field, with a considerable amount of overlap in the technologies described in those patents. ATP's Award Selection Process Is Unlikely to Avoid Funding Similar Research: Two factors in ATP's award selection process could result in ATP's funding research similar to research that the private sector would fund in the same time period. These two factors are inherent in the review process and limit the information the reviewers have on similar private sector research efforts. Due to conflict-of-interest concerns, technical reviewers are precluded from being directly involved with the proposed research, making them less likely to know about all the research in an area. Also, the information available about private sector research is limited because of the private sector practice of not disclosing research results. Until a patent is issued, a private sector firm generally publishes very few details about the research to protect proprietary information. Therefore, it is difficult for the reviewers to identify other cutting-edge research. ATP's Conflict-of-Interest Provision Limits Its Ability to Identify Similar Research: ATP selection officials rely on outside technical reviewers to evaluate a proposal's scientific and technical merit. All reviewers must certify that they have no conflicts of interest. To minimize possible conflicts of interest, the technical reviewers are generally federal government employees who are experts in the specific technology of the research proposal but are not directly involved with the proposed research area. Although this approach helps to guard against conflict of interest, it has inherent limitations on the program's ability to identify similar research efforts. The technical reviewers rely on their own knowledge of research underway in the private sector. One of the technical reviewers we interviewed said that he did not personally know of other companies that were doing similar work. However, he believed that it was unlikely that there were not dozens of others working on the same issue. Proprietary Information Limits ATP's Ability to Identify Similar Research: ATP reviewers are significantly limited in their ability to identify similar research efforts by an inherent lack of information on private sector research. Although ATP officials use several sources, such as colleagues, conferences and symposia, and current technical literature, to try to identify research efforts conducted by the private sector and the federal government, this information is often proprietary. Most of the private sector and university experts we consulted agreed that it can be very difficult to identify the specific research that private sector firms are conducting, especially considering the competitive nature of most industries. The early release of information on a company's research could be costly to the firm. If a competing firm could determine the nature and progress of another company's research, it could help the competitor to develop and commercialize an identical or higher-quality product before the other firm. At the very least, the early release of research information by a firm can give competitors an idea as to the focus of the firm's strategic plan. Thus, many firms are very careful about releasing detailed information related to research and development activities they are conducting. In conclusion, Mr. Chairman, the process ATP follows to select projects for funding is limited in its ability to identify similar research efforts in the private sector. Our retrospective look at the three ATP research projects showed that their goals were similar to research goals already being funded by the private sector. Examining the process that ATP uses to select projects, we found two inherent factors--the need to guard against conflicts of interest and the need to protect proprietary information--that limit ATP's ability to identify similar research efforts in the private sector. These two factors have not changed since the beginning of the program. We recognize the valid need to guard against conflicts of interest and to protect proprietary information; thus, we did not recommend any changes to the award selection process. However, we believe that it may be impossible for the program to ensure that it is consistently not funding existing or planned research that would be conducted in the same time period in the absence of ATP financial assistance. Mr. Chairman, this concludes my prepared statement. I would be happy to respond to any questions that you or Members of the Subcommittee may have. Contacts and Acknowledgements: For further information about this testimony, please contact Robin M. Nazzaro at 202-512-6246. Diane Raynes, Carol Herrnstadt Shulman, and Jessica Evans made key contributions to this statement. FOOTNOTES [1] GAO, Advanced Technology Program: Inherent Factors in Selection Process Could Limit Identification of Similar Research, GAO/RCED-00-114 (Washington, D.C.: April 24, 2000). [2] The Omnibus Trade and Competitiveness Act of 1988 (Pub. L. No. 100- 418). [3] Algorithm here refers to the mathematical procedures involved in recognizing writing as it is being written on a computer device. [4] IEEE Transactions on Pattern Analysis and Machine Intelligence, "The State of the Art in On-Line Handwriting Recognition" (Aug. 1990), vol. 12, no. 8. [5] A patent is a grant given by a government to an inventor of the right to exclude others for a limited time (usually 20 years) from making, using, or selling his or her invention. [6] GAO, Intellectual Property: Comparison of Patent Examination Statistics for Fiscal Years 1994-1995, GAO/RCED-97-58 (Washington, D.C., Mar. 13, 1997). [7] A fiber optic cable consists of many extremely thin strands of glass or plastic, each capable of transmitting light signals. Wavelength division multiplexing transmits separate light signals through a single optical fiber strand at different wavelengths. [8] "Biotech Bodies," BusinessWeek, July 27, 1998.