Detailed Information on the
National Institutes of Health: Extramural Research Facilities Construction Assessment

Ensure that 100% of 50 grantees have met all construction requirements.

Build capacity to conduct research by constructing or renovating 44 extramural facilities that can accommodate the proposed biomedical and behavioral research, research training, or research support to be conducted.

Ensure that 100% of 35 grantees have met all construction requirements.

Complete 48 construction or renovation of biomedical research infrastructures in order to build the capacity to conduct the proposed research.

Reduce the average annual cost per grant for travel and shipping activities related to managing construction grants by expanding the use of electronic project management tools.

Complete 30 construction or renovation of biomedical research infrastructures in order to build the capacity to conduct the proposed research.

Ensure that 100% of grantees have met all construction requirements.

Report that 95% of 179 extramural construction projects are in compliance with the post award 20 year usage requirement to conduct research.

Achieve average annual cost of managing construction grants of $36,530 per grant.

Complete 22 construction or renovation of biomedical research infrastructures in order to build the capacity to conduct the proposed research.

Complete 7 biocontainment facilities to support biodefense and emerging infectious disease research needs, including research on Category A-C Priority agents and newly emerging infectious diseases.

Report that 95% of 196 extramural construction projects are in compliance with the post award 20 year usage requirement to conduct research.

Achieve average annual cost of managing construction grants of $36,703 per grant.

Complete 9 construction or renovation of biomedical research infrastructures in order to build the capacity to conduct the proposed research.

Complete 1 biocontainment facility to support biodefense and emerging infectious disease research needs, including research on Category A-C Priority agents and newly emerging infectious diseases.

Report that 95% of 123 extramural construction projects are in compliance with post award usage requirements to conduct research.

Achieve cost savings (average annual cost per grant) for managing construction grants by expanding the use of electronic project management tools.

Report that 95% of 143 extramural construction projects are in compliance with the post award 20 year usage requirement to conduct research.

Reduce the average annual cost per grant for travel and shipping activities related to managing construction grants by expanding the use of electronic project management tools.

Complete 2 biocontainment facilities to support biodefense and emerging infectious disease research needs, including research on Category A-C Priority agents and newly emerging infectious diseases.

Ensure that 100% of 21 grantees have met all construction requirements.

Report that 95% of 164 extramural construction projects are in compliance with the post award 20 year usage requirement to conduct research.

Complete 4 biocontainment facilities to support biodefense and emerging infectious disease research needs, including research on Category A-C Priority agents and newly emerging infectious diseases.

Achieve average annual cost of managing construction grants of $36,419 per grant.

Achieve average annual cost of managing construction grants of $35,837 per grant.

Measure: Ensure that 100% of grantees have met all construction requirements, including NIH approved design and construction documents that ensures proposed research in the space is feasible, and ensures that grantees will take action to file or record a Notice of Federal Interest that ensures grantees cannot lease, sell or mortgage property without NIH approval.

Explanation:The measure reflects the percentage of grantees meeting all construction requirements.

Measure: Percent of extramural construction projects that are in compliance with the post award 20 year usage requirement to conduct research.

Explanation:This measure reflects the percentage of extramural construction projects that are compliant with the requirement to use the facility for its intended research purpose for 20 years.

Measure: Achieve average annual cost savings of managing construction grants by expanding the use of electronic project management tools that enhance oversight and 20 year usage monitoring.

Explanation:

Measure: Construct or renovate 153 biomedical research facilities in order to build the capacity to conduct the proposed research.

Explanation:

Measure: Complete 15 biocontainment facilities to support biodefense and emerging infectious disease research needs, including research on Category A-C Priority agents and newly emerging infectious diseases.

Explanation:Note that while there are facilities are under construction, none are scheduled to be completed in FY 2006.

Is the program purpose clear?

Explanation: The purpose of the National Institutes of Health (NIH) Extramural Construction grant program is to support the costs of designing, constructing and/or renovating non-Federal basic and clinical research facilities to assist in meeting the biomedical and behavioral research, research training, or research support needs of the nation.

Evidence: Specific authorizing legislation for the construction and/or renovation of research facilities is in the PHS Act. Seven NIH Institutes and/or Centers (IC) (NIAID, NCRR, NCI, NEI, NHLBI, NCMHD, and NIDA) also have the authority to make awards to construct or renovate research facilities. Two ICs (NIAMS and NIDCD) have authority to make awards to modernize research buildings. The Office of AIDS Research (OAR) plans, coordinates, and evaluates all AIDS research conducted or supported by NIH and allocates appropriated funds to the ICs. The authorizing legislation for NIAID and NCRR, the two active programs, can be found under the PHS Act section 481A(a)(1) and PL 108-7 and PL 108-276 for NIAID.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program is not able to demonstrate that its contributions are necessary to for the nation to develop sufficient capacity for biomedical research. Between 1990 and 2001, on average, funding from the NIH Extramural Construction program constituted less than 3% of total funding spent on the construction of biomedical research facilities. A biannual survey conducted by NSF describes the need for research facility space as reported by academic institutions. However, the report describes the need for research facility space as additive of what each institution reports. A better demonstration of need would describe, based on an assessment of the nation's biomedical research needs, what amount of research facility space would be necessary to conduct said research. It is not clear whether the sum of the needs reported by academic institutions is an accurate indicator of how much space would adequately support research capacity goals. Additionally, according to the NIH Working Group on Construction of Research Facilities report, contributions from other donors are at times as high as 95% of the total cost of the project. One exception is the need for biocontainment laboratories. In recent years emerging threats of bioterrorism and globalization of infectious disease have created a need for the Federal government to provide a timely response to threats. For instance, the program has focused on our Nation's fight against bioterrorism by addressing the shortage of biocontainment laboratory space.

Evidence: Descriptions of the need for infrastructure for biodefense research can be found in the NIAID Strategic Plan for Biodefense Research. NSF studies showing the total dollars spent on biomedical research facilities construction are at www.nsf.gov/statistics/facilities.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The NIH Extramural Construction grant program provides matching funds (e.g. 50%:50% or 75%:25%) as prescribed by statute to assist an institution to construct or modernize a research facility. NIH efforts are in collaboration with, complementary to, or in partnership with other organizations and industry since grantees must obtain matching funds from non-Federal sources. Applications received are in response to Funding Opportunity Announcements and are subject to peer review. During award negotiation, program and grants management staff ensure there is no redundancy or duplication of support by obtaining: 1) an assessment of the status of the project to ensure that the project has not already begun nor is complete, 2) updated financial information to ensure the availability and source of matching funds necessary to ensure completion of the project, and 3) other sources of support information for key personnel to ensure the project does not overlap with other pending applications and/or already funded projects. When opportunities arise in which collaborative efforts are needed to build a facility that has multiple purposes (such as medical services), NIH collaborates with public and private partners in order to share resources, reduce duplication, and enhance the use of the facility.

Evidence: Matching requirements are in statute in section 481A of the Public Health Service Act and in regulation at 42 CFR part 52b.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: NIH extramural construction grants are subject to grant regulation, program regulation, HHS,PHS policy, NIH's Office of Extramural Research (OER), Office of Policy for Extramural Research Administration (OPERA) established policies and procedures, and the terms and conditions of award. These requirements and processes are designed to minimize flaws. Analysis of the needs for different types of research facilities has led NIH to use two different mechanisms in making awards for facility construction projects. The NCRR Research Facilities Improvement Program (RFIP) has a one-to-one cost sharing requirement and a relatively low ratio of staff to the number of projects, primarily due to the use of the grant mechanism. In contrast, the NIAID program requires a 25% match and has a higher ratio of staff to the number of projects, due to the use of a cooperative agreement mechanism in order to be able to have more input into each project to ensure that these projects meet a specific high-priority need.

Evidence: Grant announcements available on grants.gov describe the funding mechanism and matching requirements.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program provides competitive funding through the grant or cooperative agreement mechanism to address a specific need, such as biodefense, or a broad research purpose, such as cancer, heart, arthritis, etc. The peer review process used to select grantees ensures that the most meritorius projects are supported.

Evidence: Grant announcements available on grants.gov describe the funding mechanism and matching requirements. A summary of the NIH peer review process is available at http://grants.nih.gov/grants/peer/.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has two long-term outcome measures and two long-term output measures. The outcome measures show the extent to which grantees are compliant with requirements during and after construction. Adherence to these requirements is important to ensure that funded projects stay on time, on budget, within scope, and are used for their intended purpose after construction. The output measures show the number of projects completed in each year. When considered together, these measures illustrate how the program is supporting construction of projects that will enhance the ability of academic institutions to conduct biomedical research.

Evidence: The long-term performance measures are listed below.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The targets for the program's long-term measures are ambitious. In past years, some grantees were not compliant with requirements during and after construction of the research facility. NIH responded by increasing grant monitoring, and as a result, a larger percentage of grantees are compliant. Based on recent performance, the targets are ambitious yet achievable.

Evidence: The long-term performance measures are available below.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has two annual outcome measures and two annual output measures. The outcome measures show the extent to which grantees are compliant with requirements during and after construction. Adherence to these requirements is important to ensure that funded projects stay on time, on budget, within scope, and are used for their intended purpose after construction. The output measures show the number of projects completed in each year. When considered together, these measures illustrate how the program is supporting construction of projects that will enhance the ability of academic institutions to conduct biomedical research.

Evidence: The annual performance measures are listed below.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The targets for the program's annual measures are ambitious. In past years, some grantees were not compliant with requirements during and after construction of the research facility. NIH responded by increasing grant monitoring, and as a result, a larger percentage of grantees are compliant. Based on recent performance, the targets are ambitious yet achievable.

Evidence: The annual performance measures are available below.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All performance measures for this program reflect the efforts of grantees to commit to and comply with program requirements that are designed to ensure that funds support construction of facilities that can support biomedical research. Grantees commit to these goals as a condition of the grant agreement.

Evidence: The NIH Grants Policy Statement lists the requirements to which grantees commit as a condition of their award.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Office of Director, Office of Extramural Research, has a plan to evaluate the self-certification process NIH uses to monitor the use of grant-supported space during the 20-year usage period. The plan is to conduct an independent process evaluation. The purpose of the evaluation is to validate the self-certification process to assess the effectiveness and accuracy of the self-reported data. The FY 2008 expected results will assess whether this monitoring system should continue or be redesigned. The NCRR has a plan to evaluate the effectiveness of the RFIP at fulfilling its mission to support the construction and renovation of biomedical and behavioral research facilities. The evaluation will be a two part validation assessment of Strategic Goal 2 and its annual targets, which are identified in the Extramural RFIP 2005-2009 Strategic Plan. Strategic Goal 2 focuses on the selection factors that are used when RFIP makes decisions about which applications to fund. Independent evaluations will be implemented to assess the quality of the NIAID Biodefense Research Program, of which the NIAID Extramural Biocontainment Facilities Construction Program is a part, due to the magnitude of this program and its importance in the context of homeland security. The evaluation will encompass analysis of several parameters that reflect the responsiveness and impact of the biodefense research program on the goals set in the NIAID Strategic Plan for Biodefense Research, the NIAID Biodefense Research Agenda for CDC Category A Agents and the NIAID Biodefense Research Agenda for Category B and C Priority Pathogens. The evaluations will provide feedback and opportunities for improvement of the program.

Evidence: Descriptions of these plans for evaluations can be found in the NIAID Strategic Plan for Biodefense Research, the NIAID Biodefense Research Agenda for CDC Category A Agents and the NIAID Biodefense Research Agenda for Category B and C Priority Pathogens.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The program has not requested funding in the past few years, with the exception of BSL labs.

Evidence: Evidence includes the NIH Congressional Justifications.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: All facets of the Extramural Construction program use an extensive planning process to implement the program. Corrective strategies to deal with unforeseen or unintended deficiencies regarding construction grants have been developed to address common issues. Several unanticipated situations can occur throughout the pre-construction process. For instance, early in the process a grantee may fall behind the schedule set for the design phase of the project which, for NCRR, must be completed within 24 months after the award. Each of these designs is reviewed by the NIH Design Review Team which takes 3-5 weeks for each review. If it is determined that the project is behind on the design schedule, NCRR employs corrective strategies, such as an on-site design review. Security standards for government facilities have been rapidly evolving since September 11, 2001. At the time when the NIAID Extramural Biocontainment Facilities Construction Program was first launched, there was no single source of security standards available as a reference for applicants. Because of the importance of security for these facilities in light of their role in the biodefense effort and in terms of federal investment, NIAID worked to develop a plan to ensure that security issues would be adequately addressed in the new biocontainment facilities. NIAID held a workshop for NBL/RBL applicants to discuss the NIH Physical Security Design Guidelines for NIH-Owned Facilities, which includes a requirement for each facility to conduct a Threat Risk Assessment to guide grantees on needed security measures. The Threat Risk Assessments resulted in the identification of appropriate security measures for planned facilities. NIAID responded to the need for additional funds to cover the cost of these additional security measures by issuing administrative supplements to awardees who demonstrated justified need.

Evidence: Descriptions of these policies and procedures can be found in NIH's Strategic Plans and Congressional Justifications.

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: Analysis of the needs for different types of research facilities has led NIH to use two different mechanisms in making awards for facility construction projects. The NCRR Research Facilities Improvement Program (RFIP) has a one-to-one cost sharing requirement and a relatively low ratio of staff to the number of projects, primarily due to the use of the grant mechanism. In contrast, the NIAID program requires a 25% match and has a higher ratio of staff to the number of projects, due to the use of a cooperative agreement mechanism in order to be able to have more input into each roject to ensure that these projects meet a specific high-priority need.

Evidence: Evidence include the RFAs for the NCRR RFIP and the Regional Biocontainment Laboratories Construction Program.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The program has not requested funding in the past few years, with the exception of BSL labs.

Evidence: Evidence includes the NIH Congressional Justifications.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The approved scope of the project is made a term of each award and becomes the focus of programmatic monitoring. Construction grants are subject to the requirements specified in program regulations and the NIH Grants Policy Statement (NIH GPS), which represent the general terms and conditions of award. NIH developed a core set of requirements that must be met or corrective action taken at the pre-award, award, and post-award stage. Prior to issuing an award, NIH must ensure the applicant has sufficient matching funds available, is in compliance with public policy requirements (e.g. environmental, historic preservation regulations, etc.), and has sufficient title to the site or a long-term leasehold interest in the site to ensure compliance with the 20-year usage obligation. NIH restricts the use of funds and releases funds upon completion of stated milestones. After construction begins, NIH also monitors the grantee's compliance with the filing of the Notice of Federal Interest. The release of awarded funds during the performance of the project is carefully monitored based on a demonstration of satisfactory progress in meeting each milestone, administrative and fiscal compliance and demonstration of continuing need for resources. Upon completion of the project, a final on-site inspection and cost review is performed as part of the closeout procedures to ensure compliance with the terms of the award. Monitoring is also performed by NIH staff throughout the 20-year usage obligation through NIH site visits, periodic facility usage requirement reports, publications, and other information available to NIH. When a grantee fails to comply with the terms and conditions of award, enforcement actions are applied. These may include modification to the terms of award, suspension, termination, withholding of restricted funds, or recovery of funds.

Evidence: The NIH Grants Policy Statement provides grantees and NIH staff with specific administrative requirements that affect project monitoring. The document also outlines enforcement actions taken by NIH when a grantee fails to comply with the terms and conditions of award.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: All Federal managers and program partners involved in the management, review, and administration of Extramural Construction grants have established roles and responsibilities with enforcement penalties associated with failure to meet cost, schedule and performance results as demonstrated in their performance plans and notice of grant awards. All NIH employees are required to have results-oriented performance plans or contracts which the supervisor uses to assess performance levels. Personnel performance plans for program managers at the Institutes and Centers (IC) include responsibilities for Extramural Construction Grant project oversight as appropriate. All manager positions where the assigned responsibilities are instrumental in recovering and reducing improper payments have this element incorporated in their employee performance appraisal. NIH grants management and program staff utilize reports, correspondence from the grantee, site visits and other information available to NIH to monitor cost, schedule and performance results of their grantees throughout all stages of award to identify potential problems and areas where technical assistance or enforcement action may be necessary. Poor performance results in enforcement actions that includes modification or termination or suspension of award, withholding of restricted funds, recovery of funds, or the placement of special terms and conditions on the grant award.

Evidence: Evidence includes the Draft NIH Grants Administration Manual on Extramural Construction, NIH Grants Policy Statement section on Extramural Construction, NIH Construction Grant Program Regulations 42 CFR Part 52b, and NIH Grants Manual Chapter 4.1.04.204 Responsibilities of NIH Grants Management Staff.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: When NIH issues a construction grant award, the majority of obligated funds are restricted upon award and are released by the IC when the grantee fulfills established milestones as stated in the terms and conditions of award. Grantees must reach completion of a pre-set milestone in order to be able to draw down the next increment of funding to continue the project. This milestone-driven approach ties funding to performance and provides additional assurance that obligated funds are being used for the intended purpose and in accord with the terms and conditions of award. NIH cannot support the project unless it has assurance that funds will be used appropriately, adequate documentation of transactions will be maintained, and assets will be safeguarded. Grantees must have in place accounting and internal control systems that provide for appropriate monitoring in accordance with the cost principles. In addition, their systems must be able to identify inappropriate obligations and expenditures of funds. A grantee's failure to establish adequate control systems constitutes a material violation of the terms of award which may result in modification to the terms of award, suspension, termination and withholding of support.

Evidence: Evidence includes spending reports and NIH GPS section on Financial Reports.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has developed a new efficiency measure that will show NIH's efforts to reduce the average cost per grant of managing grants by expanding the use of electronic project management tools. Additionally, NIH's extramural construction grant program has a variety of procedures and tools in place to achieve efficiencies and cost effectiveness in program execution. NIH's construction grant program has achieved efficiencies through numerous IT improvements. The goal of NIH's electronic Research Administration (eRA) system is to reduce the paper generated annually from grant application receipt through award. Through the creation of "most efficient organization", NIH has been successful in A-76 outsourcing competitions, most recently in the area of extramural activities support services, by focusing on increased efficiencies, consolidating like activities to gain economies of scale, re-engineering and creating an organizational entity for the specific purpose of monitoring the work of the new organization and identifying opportunities for increased efficiencies and effectiveness.

Evidence: Evidence includes the NIH Guide Announcement: NIH Announces Plans to Eliminate Mailing of Paper Notifications: Summary Statement & Peer Review Outcome Letters, NOT-OD-05-075, and Most Efficient Organization (MEO) Summary for NIH Extramural Activities Support Services: http://a-76.nih.gov/MEO%20Summary.pdf.

Does the program collaborate and coordinate effectively with related programs?

Explanation: NIH relies upon electronic systems to encourage appropriate collaborations and trans-agency oversight. NIH's electronic Early Notification System (ENS) ensures that opportunities for collaboration are considered across NIH programs as funding opportunity announcements (FOA) are developed. Used by NIH, the Agency for Healthcare Research and Quality (AHRQ), the National Aeronautics and Space Administration (NASA), and the Centers for Disease Control and Prevention (CDC), the ENS provides the opportunity to preview funding opportunity announcements for program relevancy and overlap before the announcements are published in the NIH Guide for Grants and Contracts. NIH assures that there is no duplication or overlap of awarded grants between agencies and institutes. Just-in-time procedures require grantees to provide active and pending support information for all key personnel at the time of the competing award. Grants management specialists work with Program Staff to resolve any issues identified for scientific, budgetary, or commitment overlap prior to award. When extramural construction grant funds are appropriated in a given year to more than one NIH Institute, Center, or the OAR, these entities collaborate by co-funding applications received under a single FOA when an application fulfills the mission of the IC or entity.

Evidence: Evidence includes the Early Notification User List, which shows system use by multiple Agencies; NIH Just-in-Time Procedures: http://grants1.nih.gov/grants/policy/nihgps_2003/NIHGPS_Part4.htm#_Toc5460005; and NCRR and NIAID joint announcement PAR-04-0122, Extramural Research Facilities Improvement Program

Does the program use strong financial management practices?

Explanation: The program does not use strong financial management practices. In the Financial Section of the HHS's FY 2005 Performance and Accountability Report, the independent auditor indicated that "HHS continues to have serious internal control weaknesses in its financial systems and processes for producing financial statements. ?? These long-standing issues, including system and process limitations and expertise needed in meeting evolving financial reporting requirements, simultaneously with implementing new systems, will require a sustained commitment and qualified support team to resolve in preparation for FY 2006 and future years." In addition, the report specifically cites that NIH launched the Oracle General Ledger portion of the NIH Business System (NBS) because its previous legacy NIH Central Accounting System was not designed for financial reporting purposes. The report also indicates that "[a]lthough the Oracle General Ledger became the official accounting system of record ?? NIH was required to record approximately 120,000 nonstandard accounting entries with an absolute value of $5.6 billion to adjust budgetary and proprietary accounts. Additionally, the NBS does not provide for tracking manual or non-routine entries. As a result, adjustments and corrections cannot be readily identified."

Evidence: Evidence can be found in the FY 2005 HHS Performance and Accountability Report - http://www.hhs.gov/of/library/par05/pdfmenu/.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: NIH has taken meaningful steps to detect and address its management deficiencies by implementing internal control processes. All NIH organizations are required to ensure that adequate systems of internal controls are developed and implemented for the proper oversight and monitoring of NIH programs. NIH's OER, in conjunction with OMA and IC representatives from the extramural business areas, developed an extramural research management controls self-assessment questionnaire to assist ICs in identifying potential areas for improved compliance with both NIH and IC specific policies and procedures. OER also has instituted a program to ensure that NIH-wide grants management policies are fully and correctly implemented by NIH ICs and that NIH ICs comply with both NIH and their own established policies and procedures.

Evidence: Evidence includes the NIH Extramural Management Controls Questionnaire, the NIH Targeted Management Controls Assessment Questionnaire for Extramural Construction, and Revised section of the NIH GPS applicable to construction grants.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The extramural construction grant program is further defined in the Catalog of Federal Domestic Assistance (CFDA). This resource provides information to the applicant community about programs available for construction and modernization projects. Similar information is also found in each individual Funding Opportunity Announcement (Program Announcements [PA] and Request for Applications [RFAs]) published daily in the NIH Guide for Grants and Contracts. NIH staff also communicates information about the program through public briefings to potential applicants. In addition, to ensure the selection of best applications with relevant need, the program has been designed to take advantage of peer review to support funding decisions. This ensures that the most meritorious applications are identified for funding. Peer review committees are composed of non-Federal scientists and other appropriate experts, such as architects and engineers, in the case of construction grants. Peer review focuses on the scientific merit of the programs and/or funded research activities to be housed in the new facilities. The review focuses on the need for the facility and its impact on the research programs, the quality of the research as judged by prior peer reviews of the various research elements, and the synergistic effects.

Evidence: A summary of the NIH peer review process is available at http://grants.nih.gov/grants/peer/. NIH grant announcements can be found at http://grants.nih.gov/grants/guide/index.html.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: During the post-award stage, grant recipients are required by the terms and conditions of award to utilize the grant-supported space for its intended purpose for 20 years. After the project is complete, NIH staff periodically monitors the use of space through site visits, periodic facility usage requirement reports, publications, and other information available to NIH. However, NIH largely relies on self-reporting on the part of research institutions to identify whether these institutions continue to use NIH-funded facilities for PHS-supported research. Large biodefense construction projects, issued as cooperative agreement awards, have substantial programmatic involvement with the awardee during the performance of the activity and are part of the National Institute of Allergy and Infectious Disease (NIAID) Biodefense Network. For these cooperative agreements, the NIAID utilizes a Construction Quality Management (CQM) contractor to assist NIAID program staff in their role to manage projects for the construction of new biocontainment facilities. The CQM provides technical support to NIAID staff to help NIAID ensure that the research facilities are developed as approved and are fully compliant with certain design and construction quality standards.

Evidence: Evidence includes NIAID NBL/RBL Construction Grant Program; Design Submission Requirements, NIAID NBL/RBL Construction Grant Program, On-Board Review Process; and NIAID NBL/RBL Construction Grant Program, Construction Phase Oversight; and Chapter 4.1.04.204 (6/24/2002) - Responsibilities of NIH Grants Administration Staff.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Award data is made available to the public in a transparent and meaningful manner. The NIH makes grant award data available through the Computer Retrieval of Information on Scientific Projects (CRISP) system. The public can search the CRISP database to: 1) analyze current Institute portfolio, 2) locate experienced NIH funded investigators in an area of interest, 3) check out the competition, 4) retrieve a listing of current and historical awards and 5) obtain abstracts of awarded projects. In addition, NIH-funded investigators are requested to submit to the NIH National Library of Medicine's (NLM) PubMed Central (PMC) an electronic version of the author's final manuscript upon acceptance for publication, resulting from research supported, in whole or in part, with direct costs from NIH.

Evidence: The CRISP system can be found at crisp.cit.nih.gov.

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: This question does not apply to this program because it is a competitive grant program.

Evidence:

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program met each of its outcome and output performance targets in FY 2005, the most recent year for which final data is currently available. In past years, some grantees were not compliant with requirements during and after construction of the research facility. NIH responded by increasing grant monitoring, and as a result, a larger percentage of grantees are compliant.

Evidence: The performance measures and data are available below.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program met each of its outcome and output performance targets in FY 2005, the most recent year for which final data is currently available. In past years, some grantees were not compliant with requirements during and after construction of the research facility. NIH responded by increasing grant monitoring, and as a result, a larger percentage of grantees are compliant.

Evidence: The performance goals and data are available below.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has developed an efficiency measure that will show NIH's efforts to reduce the cost of managing grants by expanding the use of electronic project management tools to reduce the average annual management cost per grant. The program has achieved some efficiencies by working with grantees to ensure that they adhere to schedule and cost targets. The output measures show that in FY 2005, the most recent year for which final data is available, the program completed all projects on schedule.

Evidence: Evidence includes the data for the efficiency and output measures below.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The NIH Design Policy and Guidelines (DPG) is a recognized industry standard for research facilities. The American Institutes of Architects (AIA), Center for Advanced Technology Facilities Design used NIH's DPG as a starting point for developing the AIA's Guidelines for Planning and Design of Biomedical Research Laboratory Facilities. The NIH Office of Research Facilities (ORF) collaborates on bioenvironmental research to improve indoor air quality and save energy in biomedical laboratories and animal research facilities. The results of these studies have been cited and, in many cases, adopted as national and international standards for labs and hospitals. Among the standard-setting organizations and institutions that adopted or use ORF applied facility studies are: American National Standard Institutes (ANSI), American Society of Heating and Refrigeration, and Air Conditioning Engineers (ASHRAE), American Institute of Architects (AIA), Environmental Health and Safety Department, Harvard University, University of Maryland, Massachusetts Institute of Technology, Oklahoma State University, International Conference on Indoor Air Sciences, International Conference on Healthy Buildings, and Guide for Care and Use of Laboratory Animal. Due to the uniqueness of this program in its purpose, scale, and implementation strategy, there is no single program against which the NIAID Extramural Biocontainment Facilities Construction Program can be compared for a 1:1 evaluation.

Evidence: The NIH design policy and guidelines are available at http://orf.od.nih.gov/PoliciesAndGuidelines/DesignPolicy/.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Comprehensive, independent, external evaluations of this program have not yet been conducted, but are being planned.

Evidence: Descriptions of these plans for evaluations can be found in the NIAID Strategic Plan for Biodefense Research, the NIAID Biodefense Research Agenda for CDC Category A Agents and the NIAID Biodefense Research Agenda for Category B and C Priority Pathogens.