Detailed Information on the
Office of Health Affairs: Medical Coordination Assessment

Project BioShield should develop and publish a targeting methodology that explains how it will focus on the highest threat, risk, and vulnerability areas.

Develop meaningful quantitative validation of its cost efficiencies by end-Q3 and deploy them in terms of efficiency measures as well.

Develop annual and long-term performance measures that encompass the emerging mission and will develop at least one efficiency measure for the program.

Measure: Percent progress toward establishing the capacity to identify and characterize threats, and plan for and deploy targeted defenses against bioterrorist attacks.

Explanation:

Measure: Number of agencies participating in the Integrated Medical Readiness Network (IMRN)

Explanation:

Is the program purpose clear?

Explanation: The purpose of the Medical Coordination program is to synchronize the analysis, communication, and planning of all activities within DHS that deal with medical preparedness and consequences of natural disasters or terrorist attacks. The program has three distinct components: the Chief Medical Officer, Biosurveillance, and Biomedical Countermeasures. The Chief Medical Officer provides the core knowledge base and expertise to analyze the medical information, manage medical programs, and consolidate input to direct actions required to deal with threats in order to minimize medical consequences. The Biosurveillance program provides information regarding threats and the Biomedical Countermeasures program is designed to prepare to counter those threats. As the principal advisor to the Secretary for medical issues within the Department, the Chief Medical Officer (CMO) identifies and coordinates DHS agencies and components that have medical assets, personnel and/or equipment, and that deploy those assets either in support of their own missions or as part of a larger DHS response to an incident of national significance. CMO provides oversight of the Biodefense Countermeasures program, which establishes the capability to rapidly identify and characterize threats, and plan for and deploy targeted defenses against bioterrorist attacks. Further, CMO provides oversight of the National Biosurveillance Integration System (NBIS) sub-program that has adopted the mission of the National Biosurveillance Group (NBSG), which is to provide decision-makers early recognition of biological events of potential national significance, to include natural disease outbreaks, accidental or intentional use of biological agents, and emergent biohazards through the acquisition, integration, analysis, and dissemination of information from existing human disease, food, agriculture, water, meteorological, and environmental surveillance systems and relevant threat and intelligence information.

Evidence: Project BioShield Act of 2004; Homeland Security Act (HSA) of 2002; DHS Strategic Goal 3, Objective 3.7; FY2006 Congressional Justification; Preparedness Directorate Budget in Brief; Future Years Homeland Security Program (FYHSP) system; The NBIS sub-program has established the following long-term performance goal, which is directly and clearly aligned to the mission statement: "Function as the lead agency in the development and operation of the National Biosurveillance Integration System (NBIS) to detect biological and chemical attacks and coordinate the real-time integration of bio-surveillance data with threat information and recommended responses."; Statement of Under Secretary George W. Foresman, Under Secretary for Preparedness, Department of Homeland Security, Before the Committee on Homeland Security Subcommittee on Emergency Preparedness, Science and Technology, U.S. House of Representatives, March 8, 2006; Testimony of Dr. Jeffrey W. Runge, Chief Medical Officer, Before the House Homeland Security Committee Subcommittee on Management, Integration and Oversight - October 27, 2005; Future Years Homeland Security Program (FYHSP) system; FY2006 Congressional Justification Chief Medical Officer Extract

Does the program address a specific and existing problem, interest, or need?

Explanation: CMO is charged with coordinating Department medical preparedness and response activities, including coordination of DHS' role on pandemic/avian flu. The CMO ensures internal and external coordination of all medical preparedness activities to prevent and mitigate biologically-based attacks on human health or our food supply. Project BioShield provides new tools to improve medical countermeasures protecting Americans against a chemical, biological, radiological or nuclear (CBRN) attack. Project BioShield is a comprehensive effort overseen jointly by the Department of Health and Human Services and the Department of Homeland Security, and involving other Federal agencies, including the National Institutes of Health, as appropriate. Specifically, Project BioShield will: Ensure that resources are available to pay for "next-generation" medical countermeasures, Expedite the conduct of NIH research and development on medical countermeasures, and Give FDA the ability to make promising treatments quickly available in emergency situations. Additionally, Homeland Security Presidential Directive (HSPD) - 9 tasks the Secretary of Homeland Security to coordinate with Secretaries of Agriculture and Health and Human Services, the Administrator of EPA, and the heads of other appropriate Federal Departments and agencies to create a new biological threat awareness capacity that will enhance detection and characterization of an attack. Further, HSPD - 10, the National Policy for Biosurveillance, tasks the Secretary of Homeland Security to establish a National Biosurveillance Group (NBSG) that capitalizes upon existing surveillance systems that are focused on human disease, food, agriculture, water, meteorology, and the environment. The NBIS sub-program was created by DHS, and funded by Congressional line item, as described in HSPD-10.

Evidence: Project BioShield Act of 2004; Homeland Security Act (HSA) of 2002; FY2006 Congressional Justification; Preparedness Directorate Budget in Brief; Future Years Homeland Security Program (FYHSP) system; NIAID article, "Project BioShield" found at http://www3.niaid.nih.gov/Biodefense/Public/projectbioshield.htm. Homeland Security Act (HSA) of 2002; DHS Strategic Goal 3, Objective 3.7; Mission and Resource Allocation Plan are mapped through FY2012; Statement of Under Secretary George W. Foresman, Under Secretary for Preparedness, Department of Homeland Security, Before the Committee on Homeland Security Subcommittee on Emergency Preparedness, Science and Technology, U.S. House of Representatives, March 8, 2006 and Testimony of Dr. Jeffrey W. Runge, Chief Medical Officer, Before the House Homeland Security Committee Subcommittee on Management, Integration and Oversight - October 27, 2005 address CMO's specific functions; Milestones FY2007 -FY2012 are outlined in the FYHSP system; DHS is implementing the National Biosurveillance Integration System (NBIS); The NBIS sub-program is currently transitioning from being in the program portfolio of the Risk Management Division (RMD) of the Office of Infrastructure Protection to being executed as a component of the Medical Coordination program, through the Office of the Chief Medical Officer, also within the Preparedness Directorate; The RMD and the Office of the Chief Medical Officer execute the NBIS sub-program and seek to accomplish its mission, goals and objectives through the effective development and implementation of a set of identified NBIS capabilities. This is being realized through execution of the following three essential strategies: Information Exchange: o Between and among relevant biosurveillance agencies. Analysis of Biosurveillance Information: o Including interagency analysts detailed to the NBIS. Utilization of an Information Technology (IT) System: o Used to facilitate biosurveillance information exchange and analysis. The IT system is a vital part of the program, but it is designed to support biosurveillance analysis by DHS and interagency staff.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The components of the Medical Coordination program are designed so that they are not redundant or duplicative of any other Federal, state, local or private effort. The Project BioShield Act of 2004 legislation established the program as a comprehensive effort overseen jointly by the Department of Health and Human Services and the Department of Homeland Security, and involving other agencies, including the National Institutes of Health, as appropriate. The program is well-coordinated with internal and external stakeholder involvement, which is consistent the CMO mission of ensuring internal and external coordination of all medical preparedness activities to prevent and mitigate biologically-based attacks on human health or our food supply. The NBIS sub-program is also designed so that it is not redundant or duplicative of any other federal, state, local or private effort. The charter of the National Biosurveillance Group (NBSG) is to capitalize upon existing surveillance systems that are focused on human disease, food, agriculture, water, meteorology, and the environment; consistent with the mission of the CMO. The NBSG, through the NBIS, will collate, integrate and analyze the information from these existing systems with relevant threat and intelligence information and then disseminate this "all-source" information to appropriate Federal departments and agencies. The NBIS was created to fill an identified information sharing and information-processing gap. By design, the NBIS provides added capabilities and does not replace or displace any existing partner agency reporting responsibilities. Further, the NBIS program model includes processes that enable the program team (which includes Detailees from partner agencies) to identify gaps and overlaps of its sub-program activities, both within the programs themselves as well as between partner-agency programs. The NBIS does not replace the role of any interagency partner.

Evidence: Project BioShield Act of 2004; Homeland Security Act (HSA) of 2002; FY2006 Congressional Justification; Preparedness Directorate Budget in Brief; Future Years Homeland Security Program (FYHSP) system; DHS portion of the funding for biosurveillance was designated for developing the capability for the real-time integration of biosurveillance data acquired from partner Federal agencies, with relevant threat information. It was further specified that while each partner Federal agency will remain responsible for conducting and enhancing its own particular form of surveillance, biosurveillance information will also be made available in near real-time to DHS.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Project BioShield Act of 2004 legislation establishes that the program is a comprehensive effort overseen jointly by the Department of Health and Human Services and the Department of Homeland Security, and involving other Federal agencies, including the National Institutes of Health, as appropriate. The CMO program will adjust its mission space, priorities, and resource allocations based on guidance from Under Secretary so that there is better alignment between available resources and expected/required deliverables. The NBIS sub-program is currently in the development stage - initial operating capability has not yet been achieved. However, the NBIS program team has been engaged in program planning, design and development activities which include, but are not limited to the following: - Providing agency information to accomplish Interagency Familiarization with NBIS mission and operating concepts. - Developing NBIS Standard Operating Procedures. - Participating in Tabletop Exercises. - Analyzing biosurveillance information flows among partner agencies. - Developing Training Programs for National Biosurveillance Group (NBSG) members. - Reviewing Memorandums of Agreement (MOAs) with partner agencies. - Participating in NBIS-Lite and NBIS development efforts. - Defining reach-back requirements from NBIS to partner agency biosurveillance data and information systems. Based on an on-going assessment of results from these initial program development efforts, no major program design flaws have been identified which would create a significant limitation on the NBIS effectiveness or efficiency. As development efforts continue, and particularly as the NBIS-Lite pilot effort is executed, lessons-learned and process improvement ideas will be captured and factored into the NBIS development process.

Evidence: Project BioShield Act of 2004; Homeland Security Act (HSA) of 2002; FY2006 Congressional Justification; Preparedness Directorate Budget in Brief; Future Years Homeland Security Program (FYHSP) system; CMO 6 Month Program Review; As an outcome of the on-going NBIS program planning, design and development activities, several program risk issues have been identified. These include but are not limited to the following: ?? Insufficient and/or ineffective partner agency participation ?? NBIS V2.0 development delays. ?? Ability to obtain sufficient, qualified staffing, when needed to support program activities ?? IT Technical Issue - Development of automated mapping tool sets to enhance situational awareness may delay achieving NBIS Initial Operating Capability. ?? IT Technical Issue - Development of parsing techniques and algorithms may not be sufficiently mature to allow effective data correlation Further, planned program efforts are under way to bring about the development, deployment and operational effectiveness evaluation of the NBIS-Lite pilot program. In addition to enabling a more rapid ramp-up of system capabilities, this interim solution NBIS-Lite will support early identification of lessons-learned, best practices, process gaps and overlaps -- all of which will be factored into the design of the full NBIS system architecture, which is being done concurrently due to the time-critical need for these NBIS capabilities. Moreover, where certain necessary biosurveillance information is not available from a federal agency, the NBIS may enter into agreements with non-government information sources in order to fill identified information gaps.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Once implemented, the NBIS will integrate relevant threat information with real-time syndromic and health data on humans, animals, wildlife and plants, including indicators of the presence of pathogens and toxins in the environment, to aid in the response to threats and incidents. By serving as a central fusion center, the NBIS will promote information sharing and joint analysis at the local, state, and federal levels. The establishment of the NBIS will also shape future disease surveillance systems and guide research and development of follow-on technologies and capabilities. However, it is unclear that procurements under Project BioShield are targeted toward the highest threat, risk, and vulnerability areas.

Evidence: FY2008-FY2012 Resource Allocation Plans, Future Years Homeland Security Program (FYHSP) system; FY2006 Congressional Justification, Milestones through FY2007 -FY2012; NBIS Daily Report; NBIS Weekly Report

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Long-term performance goals have been developed and are outlined in the Future Homeland Security Program. Milestones have been developed through FY2012 and are identified in the FYHSP for the programs under Medical Coordination. Additional official documentation effectively captures the program's long-term performance goal, objectives and/or targets. NBIS' long-term performance measures will capture the quality and timeliness of information acquisition, data fusion and processing and reporting. Due to the newness of the program, baselines are not yet established nor results demonstrated by long-term performance measures. Although these measures are intermediate-type outcome measures until final outcome measurement data can be developed, they provide program management with a meaningful indication of the effectiveness of efforts. Continued annual improvements in timeliness of responses will be reflective of the effectiveness of process improvement efforts that are a key aspect of the NBIS program plan, during the initial years of operations.

Evidence: Performance Measures linking to DHS Planning Priorities Information Template; Future Years Homeland Security Program (FYHSP) system; FY2006 Congressional Justification; Milestones FY2007-FY2012 outlined in the FYHSP system. The FY 2006 milestones are to: Identify DHS agencies and components that have medical assets, personnel and/or equipment, and that deploy those assets either in support of thier own missions or as part of a larger DHS response to an incident of national significance; continue to integrate those components into the DHS IMRN. Identify/obtain detailed information regarding agencies' medical response capabilities. Identify external Federal, state and local partners with whom the DHS IMRN will coordinate medical responses as a consequence of incidents of national significance. In FY 2007, milestones are to: Continue identifying DHS agencies and components that have medical assets, personnel and/or equipment and integrate those components into the DHS IMRN. Maintain documentation of components/agencies in the IMRN. Identify/obtain detailed information regarding agencies' medical response capabilities and identify the types of doctrine that will be needed to reflect the mission, purpose, scope and interrelationships of the components. Begin developing doctrine that reflects the mission, purpose, scope and functions of DHS component relative to the DHS IMRN, and that is consistent with the National Incident Management System (NIMS), and the National Response Plan (NRP). Begin determining interrelationships between DHS component activities to examine and deconflict redundancies and maximize intra - Departmental synergies. Continue developing DHS IMRN doctrine. Determine the necessity of memorandums of agreement/memorandums of understanding for medical response coordination between agencies. Develop and maintain actionable, intra - Departmental reach - back capabilities for Secretary and senior DHS incident management personnel. Develop a detailed catalogue of DHS medical assets, personnel, equipment and capabilities, including measures for deployment capacities and timelines. For FY 2008, milestones are: Continue integrating DHS agencies and components relavant to the mission of the DHS IMRN. Review and evaluate the effectiveness of doctrine that reflects the mission, purpose, scope and interrelationships of the components. The DHS IMRN will develop, maintain, and support intra - DHS medical capabilities. For FY 2009 -11, milestones include: Continue the identification and integration of DHS agencies and components that have medical assets, personnel and/or equipment and integrate those components into the DHS IMRN. Review and evaluate the effectiveness of doctrine that reflects the mission, purpose, scope and interrelationships of the components. The DHS IMRN will develop, maintain, and support intra - DHS medical capabilities. Finally, in FY 2012, milestones are: Continue the identification and integration of DHS agencies and components that have medical assets, personnel and/or equipment and integrate those components into the DHS IMRN. Review and evaluate the effectiveness of doctrine that reflects the mission, purpose, scope and interrelationships of the components. Develop, maintain, and support intra - DHS medical capabilities. The DHS IMRN is fully functional and is aware of its depth of operating capacities.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Long-term performance goals have been developed and are outlined in the Future Homeland Security Program. Milestones have been developed for the through FY2012 and are identified in the FYHSP. Each of the long-term performance measures adopted by the NBIS sub-program includes a quantifiable performance target that presents a challenge to program management and partner agencies, yet are realistic and suitably verifiable. These initial performance targets will be established in cooperation with each of the partner agencies. The actual performance targets and the reliability of the performance data will be directly impacted by having both a clear and meaningful criteria for timeliness of data and information transfer. The initial NBIS architecture has been based on 14 partner federal agencies. Each agency must create or revise existing standard operating procedures (SOPs) to enable this data and information transfer with the NBIS, both incoming and outgoing. These SOPs will form the basis for establishing the ambitious, yet realistic timeliness criteria. The incoming and outgoing message logs from the partner agencies will be cross-referenced as a means to verify the reliability of the information. Identified discrepancies will be resolved wherever possible and practical. Performance measure data will be available for reporting within 3 months of the NBIS achieving initial operating capability (IOC).

Evidence: Performance Measures linking to DHS Planning Priorities Information Template; Future Years Homeland Security Program (FYHSP) system; FY2006 Congressional Justification; Milestones FY2007-FY2012 outlined in the FYHSP system; Integrated Medical Readiness Network Guiding Document

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Annual performance measures have been developed which also serve as the intermediate outcome measures explained in 2.2. and are outlined in the Future Years Homeland Security Program system. Milestones have been developed for through FY2012 and are identified in the FYHSP and FY2006 Congressional Justification. All of the major initiatives and other activities being executed within the NBIS sub-program have one or more performance metrics and performance milestones that are used to assist program management in evaluating progress toward achieving the program's long-term goals. These annual performance measures are developed as part of the annual strategic planning and program planning processes and are directly linked to proxy outcomes. They have been selected based on their expected contribution to meeting sub-program goals. These metrics and milestones are generally focused on the identified key performance areas of those activities (i.e. those activities where successful results are deemed necessary for project success) and, more specifically, are focused on results that are expected to be achieved within the current fiscal year. In some cases, the metrics or milestones are focused on products or services that are a critical input to some other activity within the sub-program itself. Both quantitative and qualitative results of progress are periodically reported to program management to enable pro-active decisions regarding corrective actions and resource re-allocations. Measures furthering supporting the CMO's emerging mission will be developed, and an efficiency measure will be developed in the next year.

Evidence: Performance measures in the Future Years Homeland Security Program system; FY2006 Congressional Justification

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program does not have baselines demonstrating results through performance measures as the program is so new. The Future Years Homeland Security Program system has specific milestones and targets for the each subprogram: Biosurveillance, BioDefense Countermeasures, and the Office of the Chief Medical Officer. Performance results are being baselined. Program contracts have specific targets, technical performance, and cost goals that are reviewed so that progression or future contracts or follow-on phases are based upon specific metrics and competitive performance in the programs. Current annual performance measures focus on key issues that will inform future NBIS program planning. As an example, the identification of process improvement ideas will inform strategic planning and future program development and enhancement planning (and budget request preparation). Likewise, identifying available sources of data and information will not only drive planning and execution efforts to establish MOAs with these entities, but will also support data gap assessment efforts. NBIS program management has established ambitious targets for these measures as part of their efforts to bring capabilities on line quickly and once on line to enhance the level of service being provided.

Evidence: The Future Years Homeland Security Program system; Program reviews

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Individual evaluations are linked to performance and accountability. Additionally, award and continuation of funding is based on lifecycle management plans and program performance. The administration of the NBIS sub-program within the Risk Management Division (RMD) has been structured to ensure that the program manager, along with the partner agencies and the contractors, are held accountable for achieving program results. The NBIS Program Plan was developed and is being maintained directly by the program team. It brings focus to the key initiatives and activities within the sub-program. It also includes the key milestone and accomplishment (output and outcome) performance level targets that the program team has determined are necessary for mission success. This Program Plan has been reviewed to ensure that it supports higher-level strategic goals and objectives and that the planned activities are properly integrated with related project plans. Cost estimates are developed by each program team to support project budget development. This on-going participation by the program team in the planning and budgeting cycle is key to achieving buy-in by the team and it forms the basis for cost, schedule and performance accountability. As noted in the response to question 2.3, all of the major initiatives and other activities being executed within the NBIS sub-program have one or more annual performance measures that are developed as part of the annual strategic planning and program planning processes. Each measure is assigned to a specific individual on the program team. These individuals participate in setting the target levels of performance for their assigned measures, and further, are held accountable for meeting those targets by the required due dates. NBIS sub-program partners are held accountable for the data provided by a memorandum of agreement between DHS and the partner agency or organization.

Evidence: BioDefense Countermeasures FY2008-FY2012 Resource Allocation Plan; staff Position Description (PD); Independent Government Cost Estimate (IGCE); Development of the OMB300; Statement of Objectives (SOO); Application of the Earned Value Management System (EVMS)

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: BioDefense Countermeasures is heavily engaged in establishing the capability to rapidly identify and characterize threats, and plan for and deploy targeted defenses against bioterrorist attacks. BioDefense Countermeasures Priorities Template document linking to the DHS priorities outlines the intention of conducting recurring assessments to ensure the appropriate medical countermeasures, medicines, and vaccines are prepurchased for the Strategic National Stockpile. The NBIS sub-program is new, still in the development phase and has yet to achieve Full Operating Capability. As such, NBIS has yet to evolve to the point where independent evaluations are a part of the business process. However, the NBIS sub-program is currently relying upon a series of self-evaluations that cover a broad spectrum of the sub-program activities. As is generally the case with evolving programs, many of the opportunities for improvement are easy to identify and can be done so by the program team members directly, without external assistance. For the NBIS sub-program, these self-evaluations have been and continue to be very effective in identifying gaps, redundancies and inefficiencies, both within the sub-programs and with other programs. Long-term strategic planning for the NBIS sub-program will take into account the need for implementing a phased approach for having independent performance evaluations conducted and for developing and implementing the corrective action recommendations resulting thereof.

Evidence: Self-evaluations include, but are not limited to: ?? Table Top Exercises and Readiness Assessments -- to "dry run" the various NBIS data collection, fusion, analysis and reporting processes for training purposes and to facilitate continual evaluation of the effectiveness and relevance of the process steps. ?? Monthly Performance Brief and Metric Scorecards used for regular and timely reporting of the status and the assessment of progress on all sub-program annual and long-term performance metrics and milestones. ?? Ad-hoc reviews of best practices and lessons-learned (as they become known) and subsequent development and implementation of associated process improvement action plans. ?? Assessment of customer feedback obtained through various communications channels.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: CMO recently completed a FY2008-FY2012 Resource Allocation Plan (RAP) for BioDefense Countermeasures, which includes an outline of the program's priority areas, budget allocation, and linkages to Department priorities. The multi-year spend plan is linked to the effective management of BioDefense Countermeasures' performance based budget. An interagency agreement has been drafted for execution between DHS/CMO and HHS to transfer DHS funds from the Special Reserve Fund to the Department of Health and Human Services in accordance with the attached Statement of Objectives and Terms and Conditions. The budget for the NBIS sub-program shows alignment of planned activities to the long-term performance goals. In addition, a Benefits statement, a Measure of Success statement and a set of Key Milestones are developed and presented for each budget activity line item. The milestones focus on key activity efforts and include associated targets for completion time and levels of performance. Quarterly performance reviews are conducted to assess whether program activities are being executed in accordance with the plans. However, because BioShield funds are provided through an advanced appropriation, there is no explicit link between budget and performance for Project BioShield.

Evidence: Statement of Objectives Supporting an Interagency Agreement between DHS/CMO and HHS. The agreement is authorized under Section 319F-2(c) of the Public Health Service Act (42 USC 243, et seq.) as amended by Sec. 3 of Project BioShield Act of 2004; BioDefense Countermeasures FY2008-FY2012 Resource Allocation Plan; priority areas and performance measures; Evidence includes the annual President's Budget Request, the NBIS portion of the Future Year Homeland Security Plan (FYHSP) and the sub-program Annual Acquisition Plan (AAP). All NBIS sub-program activities are directly linked to the sub-program mission and to the associated long-term and annual performance measures.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Strategic planning for the program efforts continue as changing technology emerges and biological advances are made; strategic planning is continuous in the BioDefense Countermeasures program. The BioDefense Countermeasures collects data on a continuous basis through research, analysis, purchase requests and invoices. Program managers review and analyze requirements, costs and quality performance. The NBIS sub-program is a development program that is now in the start-up phase. As a result of on-going strategic planning efforts, the NBIS program management recognized the need to adopt a spiral-development methodology for this program. This rather common "build a little, test a little" approach would enable the development team to deploy basic system functionality in the near term while the full set of needed capabilities evolve from this initial set, based on a logical and prioritized capability deployment concept. This strategy has lead to the NBIS-Lite concept that is being implemented concurrently with the full NBIS IT solution development effort. Further, NBIS program management recognized the need to supplement the existing staff capabilities with expertise in the areas of program controls, specifically with regard to this type of spiral-development program approach. The needed program guidance and support in the areas of budgeting and financial management have been secured through a Federally Funded Resource Development Corporation (FFRDC). Further, the Risk Management Division (RMD), as the current lead agency for the NBIS sub-program, conducts regular strategic planning sessions to review actual versus expected results and to discuss short-term plans of the various projects, initiatives and activities. The RMD Performance Brief and Metric Scorecards are used to support these sessions. Since many of the activities occurring within the RMD portfolio of sub-programs have interdependencies with other sub-program activities, close and continual coordination by program management is necessary to enable effective pro-active responses to unplanned disturbances. These regular strategic planning sessions also enable a pro-active approach to the adjustment of program plans to ensure continued suitability of efforts in light of the very dynamic homeland security environment.

Evidence: BioDefense Countermeasures FY2008-FY2012 Resource Allocation Plan; purchase requests; invoices, research and analysis supporting documentation; CMO FY2006 Mid-year Review; FY2008-FY2012 Unfunded Request Document; The NBIS sub-program is currently responding to a user need for near real-time worldwide situational awareness regarding avian flu. Data and information collection, analyses and reporting are being done in accordance with current NBIS SOPs. The lessons-learned from this specific "pilot-type" effort are being captured by the program development team, then assessed, and where appropriate are being adopted so as to refine and improve the NBIS processes prior to full scale IT solution development. In recognition to the fact that the full scale NBIS V2.0 effort is an IT solution development program, the current NBIS Program Plan includes provisions for conducting a technology refresh study. This would be done not only to ensure that best practices are being identified, prioritized and adopted into the operations, but also to identify and address issues that could potentially lead to degradation of product and service quality if corrective action is not taken.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The BioDefense Countermeasures program collects data on a continuous basis through research, analysis, purchase requests and invoices. Program managers review and analyze requirements, costs and quality performance. The NBIS sub-program is an IT solution development effort that is in the start-up phase. Program management is implementing suitable program controls that include processes for collecting and using performance information. The primary mechanism being used at this time is timely feedback from partner agencies. This feedback addresses content and quality issues regarding the current NBIS work products. It also includes feedback on the NBIS system architecture, on IT solution capability development plan and on NBIS SOPs as they are developed. This broad-scope partner agency feedback is assessed and, where appropriate, corrective actions deemed are developed, prioritized and implemented by the NBIS program team. CMO provides oversight of this program; however, the Risk Management Division (RMD), as the current lead agency for the NBIS sub-program, collects data and information regarding all of the long-term and annual performance measures on a monthly basis.

Evidence: Purchase requests; invoices, research and analysis supporting documentation; Progress on long-term performance measures is reported through the DHS Future Year Homeland Security Plan (FYHSP) quarterly and end-of-year performance reports; ; Risk Management Division (RMD) Performance Brief and Metric Scorecards

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Individual evaluations are linked to performance and accountability. Additionally, award and continuation of funding is based on lifecycle management plans and program performance. The administration of the NBIS sub-program within the Risk Management Division (RMD) has been structured to ensure that the program manager, along with the partner agencies and the contractors, are held accountable for achieving program results. The NBIS Program Plan was developed and is being maintained directly by the program team. It brings focus to the key initiatives and activities within the sub-program. It also includes the key milestone and accomplishment (output and outcome) performance level targets that the program team has determined are necessary for mission success. This Program Plan has been reviewed to ensure that it supports higher-level strategic goals and objectives and that the planned activities are properly integrated with related project plans. Cost estimates are developed by each program team to support project budget development. As the annual budgets are finalized, the program plan will be adjusted as necessary to ensure alignment of planned level of effort with actual level of funding. This on-going participation by the program team in the planning and budgeting cycle is key to achieving buy-in by the team and it forms the basis for cost, schedule and performance accountability.

Evidence: BioDefense Countermeasures FY2008-FY2012 Resource Allocation Plan; staff Position Description (PD); Independent Government Cost Estimate (IGCE); Development of the OMB300; Statement of Objectives (SOO); Application of the Earned Value Management System (EVMS)

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: CMO recently completed a FY2008-FY2012 Resource Allocation Plan (RAP) for BioDefense Countermeasures, which includes an outline of the program's priority areas, budget allocation, and linkages to Department priorities. The multi-year spend plan is linked to the management of BioDefense Countermeasures' performance based budget. An interagency agreement was drafted between DHS/CMO and HHS to transfer DHS funds from the Special Reserve Fund to the Department of Health and Human Services in accordance with the attached Statement of Objectives and Terms and Conditions; this is an executing Joint Material Threat Determination Process with DHS. The NBIS sub-program continues to make marked improvements in getting funds obligated on schedule and in accordance with their intended purposes. Earlier difficulties encountered with the procurement process are being corrected, in part as a direct result of the efforts of the Federally Funded Resource Development Corporation (FFRDC) staff that were brought in to supplement the NBIS program team staff capabilities in the area of project controls. For the NBIS V2.0 competitively-bid sourcing effort, the NBIS program team developed a Statement of Objectives (SOO) which defines the needed system capabilities. The selected contractor will be required to develop a Statement of Work which must clearly define the tasks to be executed and the specific deliverables to be provided, along with a definition and schedule for each. Each task and deliverable must be directly and clearly linked to the delivery of a specific capability or set of capabilities. Through this approach, the NBIS program team will ensure that payment is tied directly to the delivery of a capability, not just to the delivery of a specific work product. The NBIS sub-program is a funded, congressionally mandated effort. As required, this effort has clearly defined goals, objectives and performance requirements, all of which are taken into account as the Advanced Acquisition Plan (AAP), the spend plan and procurement packages are developed. These mandated NBIS effort is being managed and monitored such that resource allocations are clearly traceable to specific goals and accomplishments.

Evidence: Statement of Objectives Supporting an Interagency Agreement between DHS/CMO and HHS. The agreement is authorized under Section 319F-2( c) of the Public Health Service Act (42 USC 243, et seq.) as amended by Sec. 3 of Project BioShield Act of 2004; BioDefense Countermeasures FY2008-FY2012 Resource Allocation Plan; priority areas and performance measures, contract invoices; FY05 carry-over funds, as well as FY06 funds are in the process of being committed and obligated for FY06 efforts; the NBIS sub-program management team includes the appropriate level of Contracting Officer Technical Representative (COTR) support; Certified Invoices are Released for Payment; Risk Management Division (RMD) SOPs include the development and issuance of a weekly Contract Status Report that provides a timely indication of PR package status as well as the status of the contract development process and the contract execution efforts that evolve from those PRs. This report information is used by NBIS program management in support of their on-going efforts to ensure compliance with their program plan and achievement of target level of performance by all program partners, contractors and stakeholders.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not have efficiency data in place yet. However, interagency agreements are established and adhered to; data-driven recommendations based on sound policy are strongly considered, and purchase requests are reviewed for accuracy of volume and other signification focal points. Strategic planning for the program efforts continue as changing technology emerges and biological advances are made. The on-going, effective use of lessons-learned procedures within RMD has been a key enabler of continual improvements in the effectiveness and efficiency for many NBIS sub-program efforts. These lessons-learned activities are conducted formally, during strategic planning, budget preparation and program planning efforts. They are also routinely conducted on an ad hoc basis, generally in response to new information. This flow of information into and within the RMD-lead sub-programs continues to be enhanced by RMD outreach efforts to other agencies and to the user community, as well as by the continued development of a division-level, IT-enhanced knowledge management system. Many of the initiatives and activities that make up the core of the NBIS sub-program are new or relatively new efforts. Program management encourages team members to not only identify opportunities for improvement but to also promptly implement approved changes. One process used by the program manager for facilitating this is to establish both a "threshold value" and a "target value" for the annual performance metrics, whenever deemed appropriate. The threshold value represents a minimum level of performance expected, whereas the target value is a more aggressive level that will generally require efficiency improvements and/or program enhancements in order to reach them. The on-going, effective use of assessing requirement within CMO has been a key enabler of continual improvements and effectiveness. Each document executed within CMO lends to the program implementation aimed at ensuring efficiencies are achieved, constant growth of the program, and cost effectiveness.

Evidence: Statement of Objectives Supporting an Interagency Agreement between DHS/CMO and HHS. The agreement is authorized under Section 319F-2( c) of the Public Health Service Act (42 USC 243, et seq.) as amended by Sec. 3 of Project BioShield Act of 2004; BioDefense Countermeasures FY2008-FY2012 Resource Allocation Plan; CMO FY06 Mid-Year Review; CMO FY2008-FY2012 Resource Allocation Plan; Priority areas and performance measures, purchase requests and contract invoices; for many of the NBIS sub-program initiatives (i.e. activities which are new for the program or existing activities that have been significantly modified) the program manager will generally choose to conduct a limited or small-scale pilot effort. This "proof of concept" approach enables the program team to identify and correct deficiencies, to incorporate lessons-learned and best practices in order to improve the overall benefits and to conduct meaningful cost-versus-benefit analyses, all prior to committing to greater funding levels. The primary NBIS example of this is the NBIS-Lite effort that is being done in parallel with, but also in support of, the full NBIS V2.0 IT solution development effort. Further, RMD and NBIS SOPs require that the program manager assess the various contract types (e.g. competitive sourcing, cost plus fee) in order to identify the "best fit" contracting approach for each Procurement Package issued. By the 2nd quarter of FY06, except for the NBIS-Lite activity, all other contracted efforts for NBIS will be competitively bid. Also, as noted in the response to question 3.2, by using a high level of specificity in the SOWs regarding work to be performed, the program teams are able to factor lessons-learned into future activities, thus contributing to continual improvement and cost effectiveness.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Most of the related programs for the NBIS are the existing human disease, food, agriculture, water, meteorology and environmental surveillance systems that are being managed by the partner agencies. These systems provide the necessary data input streams for the NBIS. For the NBIS sub-program, the collaboration and coordination with partner agencies is being driven by the Memorandum of Agreement (MOA). This document sets forth the terms and conditions under which each agency will participate in the NBIS, and it further defines specific roles and responsibilities of each party. The Risk Management Division (RMD) conducts regular strategic planning and program review meetings among the division leadership, which includes the NBIS program manager. One of the key purposes of these sessions is to enable the various program managers to stay current with the status of related program efforts. A change in any one project can have an immediate and often significant impact (positive or negative) on one or more other projects. Many of the initiatives and activities within the NBIS sub-program includes outreach efforts to known and potential partner agencies. These efforts are intended to promote industry and government participation and collaboration. These outreach efforts have had a direct contribution to the identification of gaps and redundancies in the various data and information collection, integration, analysis, recommendation development and reporting efforts. This information informs the sub-program strategic planning process so that corrective measures can be identified and incorporated into future program plans.

Evidence: Statement of Objectives Supporting an Interagency Agreement between DHS/CMO and HHS. The agreement is authorized under Section 319F-2( c) of the Public Health Service Act (42 USC 243, et seq.) as amended by Sec. 3 of Project BioShield Act of 2004; MOAs have been completed and are in the approval stage for the Phase 1 principle external partner agencies, in accordance with the current NBIS Program Plan. Preliminary efforts are underway for developing MOAs for the Phase II set of planned partner agencies. The effectiveness of the NBIS is dependent upon each partner agency creating or modifying existing standard operating procedures (SOPs) to enable the necessary data and information transfer with the NBIS, both incoming and outgoing. The roles and responsibilities associated with accomplishing this are put forth in the MOAs.

Does the program use strong financial management practices?

Explanation: CMO's recently completed its FY2008-FY2012 BioDefense Countermeasures Resource Allocation Plan (RAP), which includes an outline of the program's thrust areas, budget allocation, and linkages to DHS priorities. The program's multi-year RAP is linked to the effective management of BioDefense Countermeasures' performance based budget. Administration of the NBIS sub-program is currently being transitioned out of the Risk Management Division (RMD) (within the Office of Infrastructure Protection (OIP)), to the Office of the Chief Medical Officer, which is also under the Preparedness Directorate (PD). RMD has developed and implemented standard operating procedures that facilitate effective financial management communications, both to and from OIP and PD. These procedures, although still evolving, also enable integration of RMD program management processes with the OIP and PD financial management processes that are in place. RMD financial management procedures are intended to be compliant with applicable Federal and DHS financial management regulations and guidelines and with sound financial management practices.

Evidence: CMO FY2008-FY2012 Resource Allocation Plan; staffing matrix, priority areas and performance measures; Statement of Objectives Supporting an Interagency Agreement between DHS/CMO and HHS. The agreement is authorized under Section 319F-2( c) of the Public Health Service Act (42 USC 243, et seq.) as amended by Sec. 3 of Project BioShield Act of 2004; The NBIS program manager utilizes program cost data from the Federal Financial Management System (FFMS) Reports, as developed by the PD Business Office, to validate and supplement internally generated cost data. For many of the contracts being issued and administered under the NBIS sub-program, the program manager develops an Independent Government Cost Estimate (IGCE). This IGCE is intended to support the sub-program budget development process and the contractor bid evaluation process. Further, where deemed appropriate, the program manager may include a requirement in the contractor Scope of Work (SOW) for a contractor-administered Earned Value Management Program (EVMP). This program will generally assign a quantifiable value to each contract milestone and this value forms the basis for progress (or partial) payments. It is also intended to create an incentive for the contractor to identify opportunities for improvement and once approved, to incorporate those ideas into the project business processes. The objective is to improve performance, reduce cost and/or shorten schedules. The contractor will generally issue a monthly report regarding EVMP progress.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The BioDefense Countermeasures program collects data on a continuous basis through research, analysis, purchase requests and invoices. Program managers review and analyze requirements, costs and quality performance. The Risk Management Division (RMD), as the current lead agency for the NBIS sub-program, conducts regular strategic planning sessions to review actual versus expected program results and to discuss short-term plans of the various projects, initiatives and activities. These sessions are also used as a forum for discussing issues that are having a negative impact on program performance and for identifying potential solutions. Action items resulting from these discussions are tasked to appropriate team members for follow-up and close out. Most issues raised have to do with a need for better integration of activities and improved and expanded information sharing, both within and between programs. The NBSG SOPs calls for the program to conduct semi-annual program reviews. One of the objectives of these reviews is to identify program performance deficiencies (including gaps), from both the contributor perspective and the user perspective. These reviews are intended to include participation by all partner agencies. The NBIS program team has developed and is currently implementing a Task Tracking System. This tool is designed to ensure better tracking of the many action items and process improvement initiatives that are underway within the NBIS sub-program. The RMD continues to review, adjust and integrate its management processes with those of OIP, PD and DHS at large. As these higher-level processes, policies, shared services and systems evolve, the RMD management processes, particularly as they relate to program management, are changed, not only to maintain alignment but to also take advantage of process improvements and available shared services. This is an on-going effort and is made possible through effective communications and information sharing among program managers and the business support functions.

Evidence: BioDefense Countermeasures FY2008-FY2012 Resource Allocation Plan; purchase requests; invoices, research and analysis supporting documentation; The current NBIS Program Plan includes provisions for training and readiness exercises for program staff, including the Detailees from partner agencies. This training and the exercises are intended to help bring about a "program focus" rather than a "stove-piped partner agency focus" that commonly occurs with a program team structure of this type. The NBIS program team continues to develop and follow standard operating procedures (SOPs) that address the need for effective integration of RMD internal financial management processes with those processes and procedures that have been adopted at the Directorate level. These SOPs continue to evolve as the Directorate-level procedures themselves evolve, as financial management requirements are communicated more effectively down to the Division level and as corrective actions are implemented based on previous year lessons-learned. These SOPs have also contributed to a substantial reduction in the level of effort and time required to develop and issue Procurement Requisition (PR) packages for the NBIS sub-program. The RMD leadership had recognized a lack of adequate, qualified Contracting Officer Technical Representatives (COTRs) on division staff to effectively handle the level of technical support associated with the numerous contractor activities being undertaken as part of the NBIS sub-program efforts. RMD division management responded to this deficiency by requiring key staff members to get COTR training and by selectively supplementing existing staff with qualified COTR support from external agencies. This has resulted in a recent increase of RMD staff COTRs from 2 to 7.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program does not yet have sufficient performance data to determine progress in achieving its long-term performance goals. However, current FY06 Execution Strategies have been accomplished or are on track for completion.

Evidence: As noted in the DHS Performance Assessment Report (PAR) for FY05, the target level of performance was met for only one of the three long-term performance measures for the NBIS sub-program. This "met" measure involved the identification and implementation of process improvements into the NBIS business process model. The other two measures address actual operations. However, operations of the NBIS were not achieved in FY05 due to contracting delays. This unexpected occurrence was the primary factor leading to lower than planned sub-program performance. For the long-term performance measures that are active for FY06, performance through 2nd quarter is on track for meeting the annual target levels of performance.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program does not yet have sufficient performance data to determine progress in achieving its long-term performance goals. Current FY06 Execution Strategies have been accomplished or are on track for completion. For FY06, the NBIS annual measures are focused on activities that are key to making initial, yet meaningful progress in the spiral-development approach that has been adopted by the program. These measures are aligned with the annual program plan and they include establishing MOAs with an initial set of partner agencies, acquiring biosurveillance data from existing systems, demonstrating the ability of the NBIS to integrate data and to identify and implement process improvement ideas into the program operations. Through the 2nd quarter of FY06, the NBIS program has been making acceptable progress toward achieving the annual performance goals.

Evidence: The Risk Management Division (RMD) collects performance metric and milestone data and issues the RMD Performance Brief and Metric Scorecards on a monthly basis. This information is used by division and program management to bring focus to issues of concern, to develop mitigation plans for under-performing activities and to track effectiveness of corrective actions implemented. As previously noted, the NBIS program is issuing daily and weekly reports. These reports demonstrate that the planned basic-level data collection and integration processes are in place and are functional. The initial Phase 1 set of MOAs with partner agencies are complete and are in the approval process. Initial work has started on the Phase 2 MOA effort. Numerous process improvement ideas are being submitted by team members and by partner agencies. Many of these ideas have obvious benefits, no drawbacks and are quick and easy to implement. These changes are being processed quickly and as a result, the formal tracking process has been capturing only the more noteworthy of the changes.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Many of the initiatives and activities that make up the core of the NBIS sub-program are new or relatively new efforts. As these efforts ramp up, the project team has been quickly identifying and responding to opportunities for process improvement. In addition, the structure of many of the sub-program efforts is still evolving in order to take advantage of outside lessons-learned and best practices and also to accommodate changing customer needs. Although many of the changes being made are "small" in the sense of their overall impact, there have been many of them and most have been relatively easy and quick to implement. Collectively they are leading to improvements in the effectiveness of the efforts and in the efficiency of execution, although a meaningful quantitative validation of this is not possible at this time.

Evidence: The NBIS sub-program is achieving improved efficiencies in several ways. The primary way is through the parallel development of the NBIS-Lite along with the full capability NBIS V2.0 IT solution. This approach enables the near term benefits of NBIS work products while also enabling the identification and capture of lessons-learned and best practices and the incorporation of them into the initial "spiral-development" of the full capability system. Another way that the NBIS sub-program is achieving efficiency and effectiveness improvements is by identifying and eliminating activities that have been shown to add little value or are duplicative of other efforts. The BioDefense Countermeasures and CMO portfolio have not completed its initial annual cycle and demonstrated performance on annual or long-term goals have not been fully evaluated or documented.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Expanded authorities as a result of the Project BioShield Act of 2004 allow for expedited peer review of grants to support research and development of countermeasures for agents of bioterrorism. The National Biosurveillance Group (NBSG), through the NBIS, will collate, integrate and analyze the information from existing bio-related surveillance systems with relevant threat and intelligence information and then disseminate this "all-source" information to appropriate Federal departments and agencies. The NBIS was created to fill an identified information sharing and information-processing gap. These added national capabilities that are being provided through the NBIS were generally not perceived to be a significant national need prior to the events of September 11, 2001. Since then however, biosurveillance has become a national policy which, along with the supporting homeland security strategic goals, has created the need to: ?? identify and collaborate with existing surveillance systems that are focused on human disease, food, agriculture, water, meteorology, and the environment, ?? facilitate the acquisition, integration, analysis, dissemination and management of data and information from these existing biosurveillance systems along with relevant threat and intelligence information, ?? provide near real-time indications data for both disease surveillance and bio-terrorism warnings, ?? provide decision-makers with early recognition of biological events of potential national significance, to guide policy and response decisions, ?? allow for information sharing among partners. Satisfying this set of needs has involved undertaking activities and initiatives that are new and unique, often with little or no known legacy processes upon which to build from. At this time, there are no known suitable programs upon which a meaningful comparison to the NBIS sub-program can be made.

Evidence: Project BioShield Act of 2004; Pandemic Spend Plan. The NBIS sub-program has and will continue with efforts to identify and leverage any applicable biosurveillance data collection, integration, analysis and dissemination knowledge and experience (e.g. lessons-learned, best practices, analytic methodologies and tools) that may exist and/or that is being developed within other government agencies, the private sector and the international communities. However, experience to date has shown that this biosurveillance knowledge and experience has generally been focused on specific issues rather than on the broader perspective of improved information sharing and enhanced situational awareness upon which the NBIS mission has been built.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: As noted in the response to question 2.6, regular, independent evaluations of sufficient scope and quality, regarding program performance, have yet to be conducted for Biosurveillance. Evaluations have not yet been conducted; although BioDefense Countermeasures is on target to achieve its long-term goal of establishing the capability to rapidly identify and characterize threats, and plan for and deploy targeted defenses against bioterrorist attacks. The portfolio has not completed its initial annual cycle and demonstrated performance on annual or long-term goals; therefore, the program has not been fully evaluated/documented to meet the scope of the question posed pertaining to effectiveness and achieving results.

Evidence: FY2006 Congressional Justification Accomplishments; BioDefense Countermeasures FY2008-FY2012 Resource Allocation Plan; drafted and/or executed Interagency Agreements