Detailed Information on the
Strategic National Stockpile Assessment

Explicitly tie budget requests to the accomplishment of annual and long-term goals and present resource needs in a complete and transparent manner.

Implement BARDA requirements from Pandemic and All-Hazards Preparedness Act of 2006

Develop and implement an Activity Based Costing Model to improve cost analysis, evaluation, planning and financial management of the SNS.

Conduct a meaningful, credible analysis of alternatives that include trade-offs between cost, schedule, risk and performance goals and use the results to guide the activity.

Manage program by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals.

Measure: Number of successful annual exercises and number of active response units for delivering assets in response to multiple events within 12 hours

Explanation:DSNS will exercise its response to multiple events annually. DSNS will also grow its TARU to support multiple responses. 12-hour response includes any appropriate response to an event. Targets set for exercise and TARU (respectively).

Measure: Number of treatments/prophylaxis for the appropriate response to known terrorist threats or public health emergencies for chemical, biological, radiological and nuclear threats in millions.

Explanation:DSNS will grow and maintain treatment/prophylaxis capability for chemical, biological, radiological/nuclear threats. Targets set for each (respectively).

Measure: Number of trained and ready Technical Advisory Response Units (TARU) for response to multiple events.

Explanation:TARU support program response. Growing the number of TARU teams will allow DSNS to more effectively respond to multiple events.

Measure: Dollars saved per $1 invested in the Food and Drug Administration's Shelf Life Extension Program (SLEP) for available projects.

Explanation:The return on investment means for every $1 spent on SLEP costs (testing, shipping, relabling, etc.), DSNS avoided $22 in replacment costs (baseline). Calculated based on the total estimted replacement costs divided by the total estimated SLEP costs for 10 FDA SLEP projects submitted by DSNS in 2004.

Measure: Percentage of inventory discrepancies that are reduced by using quality inventory management systems.

Explanation:This is an new measure for DSNS. Data had not been readily available to track inventory discrepencies in the past. Thus, the basline is under development and the performance targets may change. DSNS's new inventory management system will provide the capability to track discrepencies in the future.

Measure: The number of successful annual exercises that test response to multiple events with a 12-hour response time.

Explanation:Exercises are used to demonstrate the ability to respond.

Is the program purpose clear?

Explanation: The Strategic National Stockpile at the Centers for Disease Control and Prevention (CDC) has a focused and well-defined mission to ensure the availability and rapid deployment of medical assets and countermeasures to the site of a terrorist attack or other national public health emergency. The Stockpile is designed to provide States the medicines, vaccines, and supplies to treat those affected by a bioterrorism attack or other public health emergency and prevent the further spread of disease. The purpose of these materials is to help provide for the emergency health security of the US population in such an event. The Stockpile itself is a national repository of antibiotics, chemical antidotes, life-support medications, IV administration and airway maintenance supplies, and medical surgical items. To facilitate security and rapid deployment, these materials are placed at strategic locations around the United States.

Evidence: The program's mission has evolved along with its legislative history. Evidence includes the initial appropriation (PL 105-277) for pharmaceutical and vaccine stockpile activities at CDC known as the National Pharmaceutical Stockpile; the Public Health Security and Bioterrorism Preparedness and Response Act (P.L. 107-188), which expanded the Stockpile's role to include a broad range of public health emergencies, such as natural disasters or an influenza pandemic; the Homeland Security Act of 2002 (107-296), which required the Department of Homeland Security to define the goals and requirements of the Stockpile and manage its assets; the Project BioShield Act of 2004 (P.L. 108-276), which returned management of the Stockpile to HHS; Presidential Decision Directive 62; Homeland Security Presidential Directive/HSPD-8; HHS' operating plan for the anti-bioterrorism initiative; DHS Target Capabilities List; Interim National Preparedness Goal; and Stockpile mission statement and related documents.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program addresses the ongoing problem of threats from intentional release of smallpox, anthrax or plague, radiological/nuclear attacks, chemical attacks and other threats such as an influenza pandemic. The program addresses the problem of needing access to medical countermeasures for these threats quickly. Finally, the program addresses the need of State and local governments to obtain assistance in providing these countermeasures to respond to terrorist attacks and other national public health emergencies. For reasons of cost and efficiency, the private healthcare system does not hold large excess quantities of pharmaceuticals and other medical countermeasures that would be needed in a large-scale emergency. Local medical distributors likewise hold sufficient inventory to meet routine market demand. Material in the Stockpile is provided when there are no other sources to support a national emergency.

Evidence: Emergencies that may affect the public health for which the Stockpile is designed to address include terrorist attacks from weapons of mass destruction, including chemical, biological, radiological, nuclear and explosive; natural disasters such as earthquakes and hurricanes; technological or industrial accidents such as the release of a toxic gas; and other public health emergencies such as an influenza pandemic. Events where material from the Stockpile were deployed include the September 11, 2001, attacks on the World Trade Center and the October and November 2001 anthrax attacks. The program supports national special events such as a G-8 conference meeting held in the United States, presidential inaugurations and funerals, major presidential nominating conventions, and the State of the Union address by deploying technical advisors and tailored packages of material to the events in advance.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program is the only civilian medical stockpile for the US population. For example, the Stockpile holds enough smallpox vaccine for the entire US population. The Stockpile is designed to supplement and resupply State and local public health agencies in a public health emergency. Private healthcare systems may hold surplus medical material and many State and local governments maintain limited caches of countermeasures. The Department of Agriculture maintains a small stockpile of drugs for large disease outbreaks in animals. The Departments of Defense (DoD) and Veterans Affairs (VA) maintain stockpiles of key drugs, such as oseltamavir, for their populations. The US Postal Service and some US companies maintain small caches for their employees. Project BioShield purchases medical countermeasures for storage in the Stockpile. CDC's National Immunization Program maintains a stockpile of pediatric vaccines for routine vaccinations. None of these stockpiles meet the objectives of the program.

Evidence: States and localities are required to establish small caches of chemical antidote and biological agent countermeasures under the Health Resources and Services Administration's hospital preparedness grant and the Metropolitan Medical Response System program. These caches are designed to help first responders and provide protection until material from the Stockpile arrives. Evidence also includes the Public Health Service Act.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that another approach or mechanism, such as a mandate to State and local governments, would be more efficient or effective. As is described in Sections II and III, some management changes related to the program design are needed. There are multiple strengths in the program's design. For example, the storage and distribution network is made up of facilities from VA, DoD and the private sector in secure locations near major transportation hubs around the country. This design enables the program to provide push packages anywhere in the country within 12 hours and to provide material from alternate sites if one location cannot be used at a given point in time. The program's storage and distribution network is made up of 12-hour Push Packages, which are pre-assembled formularies of materiel in specialized containers, managed inventory, which is a range of items to backup push packages or deliver directly when the specific needs are known, vaccine repositories and smaller amounts of vendor managed inventory.

Evidence: The Stockpile was initially managed by CDC's National Center for Environmental Health and now is managed by the Division of the Strategic National Stockpile within CDC's Coordinating Office of Terrorism and Preparedness Emergency Response. The program has four branches, including logistics, response, program preparedness, program coordination, and a science team with a total of 52 Federal employees and 177 contractors. The Department of Health and Human Services' Office of Public Health Emergency Preparedness shares in the oversight of procurements and maintenance and facilitates cooperation with the Department of Homeland Security (DHS) and VA. VA's National Acquisition Center (NAC) contracts the purchase, storage, quality control, maintenance, and contingent deployment of medical materiel at a reduced cost to the program. The Logistics Managment Institute (LMI) found the program's network will cover over 99 percent of the US population within 12 hours.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: There are multiple aspects for this program in effectively targeting resources to reach intended beneficiaries. These aspects include configuring the materiel in a way that is most usable in an emergency, making appropriate decisions on when and how to deploy materiel, providing the materiel in a short period of time without fail, and working with State and local entities to ensure they are able to use and manage the materiel once delivered through planning and exercises in advance of an event and technical assistance during an event. The program is focused on each of these areas. Providing materiel from the Stockpile within required timeframes relies on State and local governments. Responsibility for working with State and local governments is shared with other CDC emergency preparedness programs. The Stockpile conducts monthly exercises to ensure CDC's ability to deploy the materiel and periodic exercises with partners to ensure partners fulfill their roles. An LMI review found internal differences in opinion over whether the program beneficiary is local governments or the public.

Evidence: To receive assets from the Stockpile, the affected State governor's office requests deployment from CDC or HHS. Federal officials primarily from HHS and CDC evaluate the presence of an emergency and the request to quickly determine whether to deploy materiel. Unused materiel following an emergency deployment is returned to the Stockpile. The program's transportation design relies on FedEx and UPS, which have redundant capability in their business plans and contractually guarantee to move these shipments first if they do not have a spare aircraft. The program also maintains priority to take off, fly and land with clearances from the Federal Aviation Administration. If ground delivery is preferable, the program also maintains a fleet of trucks. The program maintains Technical Advisory Response Units of seven to ten trained personnel that are deployed to accompany shipments from the Stockpile to advise local authorities about receiving, distributing, dispensing, apportioning, repleneshing and recovering materiel.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program adopted two new long-term measures that are meaningful and capture the program purpose. The first measure captures the amount of treatment or prophylaxis coverage capability for chemical, biological, radiological and nuclear threats. The second measure captures the delivery time as measured by the number of successful annual exercises and the establishment of active Technical Advisory Response Units (TARUs) that accompany Stockpile assets. TARUs are teams of trained individuals available 24 hours a day, seven days a week and establishment of TARUs can better enable the Stockpile to respond to multiple events. A separate CDC program maintains the performance measure on the distribution of assets from the Stockpile to the public through State and local governments. These measures capture the program's readiness to respond to a bioterrorist attack or other public health emergency. These events are not regular occurences and there are no data or regular points in time to measure the outcome of the program in lives saved.

Evidence: The first measure is the amount of pharmaceuticals and other supplies for providing an appropriate response to known terrorist threats or public health emergencies for chemical, biological, radiological, and nuclear threats. The program also maintains medical supplies that can be used for many purposes and supplies for conventional explosives and other threats. The second measure is the proven capability to deliver Stockpile assets in response to multiple events within 12-hours as measured by the number of successful annual exercises and the number of active response units.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has baselines and appropriate end targets for its long-term measures.

Evidence: The program's target for the first measure is 2.3 million, 60 million, 100 thousand, 65 thousand, respectively, in 2008. The program's target for the second measure is one event and nine TARUs in 2009.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program adopted four annual performance measures that contribute to completion of the long-term outcomes. The first measure focuses on annual exercises for delivery and response time. The second measure focuses on the number of trained and ready TARUs. These two measures relate most closely to the second long-term measure of readiness. The third measure focuses on the amount of discrepancy between what is expected and what is realized in the Stockpile inventory. This measure relates most closely to the first long-term measure of the robustness and completeness of the arsenal of countermeasures in the Stockpile. The fourth measure is the efficiency measure that focuses on savings through the Food and Drug Administration's Shelf Life Extention Program (SLEP) for materials that are potentially available for shelf-life extension. By extending the shelf-life of a product, the program does not need to spend funds to replace these products as quickly or as often.

Evidence: The program's measures include the number of successful annual exercises that test response to multiple events with a 12-hour response time; the number of trained and ready Technical Advisory Response Units (TARU) for response to multiple events; the percentage of inventory discrepancies that are reduced by using quality inventory management systems; and dollars saved per $1 invested in the Food and Drug Administration's Shelf Life Extension Program (SLEP) for available projects.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has ambitious targets and timeframes for its annual measures.

Evidence: The program's target for the first measure is one successful test in 2007. The program's target for the second measure is seven TARUs in 2007. The program's target for the third measure is less than 5% discrepancy in 2007. The program's target for the fourth measure is $26 saved per $1 invested for available projects.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Program partners, including State and local health departments, transportation carriers, VA and DHS, largely commit to and work toward the program's long-term goals. The program has a memorandum of agreement with the VA's National Acquisition Center for growing and maintaining the Stockpile. The program has contracts with FedEx and UPS to ensure the 12-hour Stockpile delivery capability is maintained. The program works with State and local governments to conduct joint exercises of the ability of the program to deliver and guide and the State and local authorities to rapidly distribute materiel from the Stockpile. The program works with partners in CDC and HHS on the Cities Readiness Initiative to establish distribution capabilities in key cities.

Evidence: The agreement with VA covers the purchase, storage, quality control, maintenance and deployment of materiel from the Stockpile. Transportation partners must, for example, be able to load one 50 ton shipment of 130 push package containers onto a wide body aircraft in approximately 40 minutes and have reserve aircraft that are crewed, fueled and ready to take off, if needed. The program exercises the transportation partners annually for delivering materiel and quarterly for delivering the Technical Advisory Response Units. State-based exercises are conducted roughly ten times per year, though multiple States are not yet ready to conduct an exercise (GAO-03-373). The program participates in some of these exercises to also provide training for Stockpile staff. Evidence also includes documentation of exercises conducted with State partners to test a State's ability to respond to various scenarios.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program will obtain a comprehensive evaluation from Clifton Gunderson, LLP that is beginning in Spring 2005. The Government Accountability Office has reviewed multiple aspects of the Stockpile. The HHS Office of Inspector General has conducted audits of physical security and environmental controls and assistance to States and cities. The OIG is currently conducting a follow up audit of physical security and environmental controls and is conducting an evaluation of the CHEMPACK project. Science Applications International Corporation (SAIC) audited the program's compliance with Good Manufacturing Practices. The program has worked with Logistics Management Institute (LMI) for multiple reviews, such as a review of the push package distribution network. The program provided the data and information for the review and LMI analyzed the information and developed the conclusions and recommendations.

Evidence: Evidence of completed evaluations include GAO reviews (B-286167, GAO/T-HEHS/AIMD-00-59, GAO 01-463, GAO/HEHS/AIMD-00-36, GAO-01-915, GAO-01-463); HHS OIG financial and programmatic reviews covering issues that include financial and environmental controls, use of aircraft, State and local preparedness and Stockpile program technical assistance; KPMG financial and inventory management system audits; and contracted reviews through LMI and also SAIC on issues that include logistics, selection of external vendors, purchasing and inventory validation, organization, CHEMPACK, materiel management, management assessment, physical security assessment. Ongoing reviews include OIG reviews of physical security and environmental controls (A-04-04-01005) and CHEMPACK distribution (OEI-04-05-00040), an independent assessment of the Stockpile's storage and distribution network by LMI. FDA will also conduct a cGMP compliance review.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: While the program has made some progress in this area, it has not yet met the criteria specified for this question to show resource allocation decisions are made in order to accomplish specific targeted performance levels and the effects of funding on results. Budget requests describe principal activities, including key objectives such as adding 10 million treatments of anthrax antibiotics to reach a treatment capacity objective of 60 million. As is described below, the CDC and the program are developing a process to more closely tie performance goals with the allocation of resources. The program also works to ensure internal requests for funding are tied to a program objective.

Evidence: Evidence includes the GPRA plans and reports and annual Congressional Justifications and budget documents provided to OMB. Additional evidence includes documentation of CDC's new goals management process.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The strategic planning deficiencies identified in this section include not fully tying budget requests to the accomplishment of performance goals and not conducting full analysis of alternatives and trade-offs in procurements and other expenditures. CDC is taking multiple steps to better integrate budget and performance information, including a process of developing cross-cutting, agency-wide goals. The program is in one of four CDC "pioneer areas" for integrating performance measures with spend plans and budget requests. CDC is introducing a new approach to ceiling letters to advance budget and performance integration through a web-based collection tool. The system is designed to allocate resources to align with performance goal areas, in addition to providing spend plan data by object class and activity area. The program is working with CDC OD to align program activities and budget resources to newly identified goals. The program also has plans to establish a more routine, scientifically-based process for informing and directing procurement decisions.

Evidence: CDC's Office of Strategy and Innovation and Financial Management Office are working to better align program activities and resources to newly developed goals. Preparedness is one of the first areas of emphasis. The program established an intragovernmental committee that can help identify the most effective materiel to include in the Stockpile. Evidence also includes updates on the OSI and FMO ceiling letter data collection tool.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: The program has not established a scientifically-based, organized, multi-year decision making process for procuring assets that incorporates threat information and is driven by a comprehensive, strategic level of readiness. While procurement objectives will change with shifting threat scenarios, the program does not have a baseline recommendation and has not developed an analysis of alternatives for new procurements. The program has not consistently followed a routine process for conducting analyses of alternatives that considers trade-offs between cost, schedule, and performance goals and does not have a process to systematically define, prioritize and budget for requirements. The program's expert medical panels are advisory and do not set firm requirements on the program. The program does take cost, schedule and performance into consideration in making decisions and has developed business cases (Exhibit 300) for select items including IT projects and assessed trade-offs in some areas, including surveys of potential facility vendors.

Evidence: Evidence includes the program's experience in making large procurements, including in some cases where final decisions have been made outside of the program's procurement planning structure. An LMI review found accountabilities for managing functional requirements are ambiguous with no single entity responsible for managing the process for defining and prioritizing Stockpile requirements. The lack of clear requirements management was found to be a risk to meeting current and future requirements affordably and responsively. Evidence of where a study of trade-offs was conducted include a February 2005 presentation on the new facility assessment and selection process for storage, location and transportation trade-offs to extend coverage under more aggressive response times. LMI conducted a cost benefit analysis of using a shelf-life extension program for CHEMPACK, compared the costs and risks for protecting local fire departments' chemical decontamination equipment in Push Package sites, and studied options related to meeting new smallpox vaccine storage objectives.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program conducts regular reviews and exercises to monitor and improve performance. The program is using findings from the LMI review to determine the best configurations for coverage, redundancy and resiliency under alternate delivery time goals. Based on this review and other performance information, the program seeks to replace existing networks that are designed by area (push packages, stockpile managed inventory, vaccines) into a multiuse network for gains in performance and efficiency. The program has supported reviews of the storage and distribution network to determine performance benchmarks and lay out decision making tools.

Evidence: The program gauges performance of distribution by the number of people, cities and reception, storage and staging (RSS) facilities that could be reached within a specified time period. The Stockpile established a Program Evaluation Unit in 2004 to conduct internal evaluations of all aspects of the program and focus on regulatory compliance and quality assurance. The unit has, for example, conducted a root cause analysis of temperature variations during storage of a vaccine. The program also established a quality control unit to ensure FDA regulary compliance. Exercises highlight weaknesses and failures in the system and are followed by highly detailed after action reviews and remediation, including periodic exercises for deploying TARU teams and Stockpile materiel. The exercises provide performance information for continuous improvement and training for staff. The program's preparedness branch also assesses the CDC cooperative agreement and participants in the Cities Readiness Initiative to determine each State's readiness to use the Stockpile in an emergency.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: In addition to multiple GAO and OIG reviews, the program will undergo a management review by Clifton Gunderson to review accountability issues. The program works with partners to ensure compliance with current Good Manufacturing Practices and conducts regular audits of Stockpile storage locations to ensure compliance. The program has also introduced additional accountability in contracts and agreements. GAO found improvements in the program's agreement with VA with procedures for unannounced inspections and remedial actions with timeframes. The program structure allows clear lines of reporting throughout the organization to senior management to ensure accountability by area. Performance ratings for management officials are linked to program objectives. An LMI review that included oversight found the program has mechanisms to hold program managers accountable but decision making authority was not fully given to the program but was held at the agency level.

Evidence: Evidence includes program procedures for corrective and preventive action for addressing deviations and policy documents and checklists for quality assurance/control site visits for the push packages, VMI and SMI and vaccines; after action reports of exercises; air cargo charter agreements; copies of memorandums of agreement with VA, USAMMA and JRCAB, and the Philadelphia Defense Supply Center; a storage contract; OIG A-04-02-08002; GAO/T-HEHS/AIMD-00-59; copies of facility audits; manager workplans. An OIG review from FY 1999-FY 2002 found partner organizations have not always carried out responsibilities effectively and the program did not fulfill its responsibilities to provide oversight and direction. The program's response documented subsequent corrective efforts to address and correct deficiencies. GAO (GAO-01-666T) also found a lack of signed agreements between CDC and their distributors and carriers. Since that time, the program has entered into contracts with distributors and carriers that more clearly lay out their responsibilities.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds for the program are obligated in a timely manner and spent for the intended purpose. At the agency level, CDC consolidated budget execution functions in 2004 into a central office that is now charged with quality assurance and data validation for program execution. The agency approves internal reallocations that vary from spend plans; regularly reviews unliquidated obligations; and established a standard operating procedure for spending plan execution to help ensure program funds are obligated consistently with the overall objective of the program and that allotted funds are fully executed in a timely manner. CDC conducted risk assessments to determine whether specific programs were susceptible to improper payments exceeding $10 million and a 2.5 percent error rate and will estimate improper payments. At the agency level, data validation of commitments is used to help identify whether funds are committed for the alloted purpose and done correctly to track with budget and accounting systems.

Evidence: Evidence includes operating procedures of the budget execution branch at CDC, sample data validation reports, the budget execution standard operating procedures, agency procedures for development and submission of annual spending plans, budget execution spending plans and obligation reports, CDC's February 2005 submission for risk assessments under the Improper Payments Information Act. OIG (A-04-02-08002) cited problems in ensuring funds were expended in accordance with applicable statutes, regulations and policies, including for air travel, a new inventory control system, and seemingly unrelated procurements. The program provided a description of contested activities to OIG and has enhanced expenditure controls.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has adopted an efficiency measure and has other procedures to measure and achieve efficiencies and cost effectiveness in program execution. The program's efficiency measure is focused on reducing the cost of replacing inventory by extending the shelf life of products that remain efficacious after their original manufacturer's expiration date through a Shelf Life Extension Program (SLEP). By extending the shelf-life of a product, the program does not need to spend money to replace that product as quickly or as often. The program's assets now include large amounts of antibiotics that cannot be rotated back into the market and chemical/nerve agent with no commercial use. Where possible, the program rotates stock in the market to avoid expiration of supplies through its vendor managed inventories. Through the program's agreement with VA's National Acquisition Center, the program achieves regular and significant savings on the purchase of medical assets. The program has also updated software used to manage inventory and finances to improve oversight and gain efficiency.

Evidence: The program's annual efficiency measure is dollars saved per dollar invested in the Food and Drug Administration's Shelf Life Extension Program (SLEP) for available projects. The program has conducted a cost analysis of multiple prior procurements for shelf-life extension, including the cost to test, repackage, re-label and transport compared to the cost of earlier replacement of each project. Competencies the program has sought commercially or through public partners instead of trying to build up its own capacities include commercial transportation, security, distribution and certain types of procurement.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The overall concept of the Stockpile is a Federal, State and local partnership where the program delivers required medical materiel to a State-designated destination within a specified time period. The program collaborates with State and local entities. The program collaborates with other Federal agencies through an intergovernmental committee and multiple memorandums of agreement. The program collaborates with the Department of Defense's Supply Center Philadephia (DSCP) and with the US Army Medical Material Agency (USAMMA) to coordinate purchase requirements, pursue the optimal contract for all members and allocate production capacity where needed. The program collaborates with the Metropolitan Medical Response System to help State and local governments prepare for receiving Stockpile materiel. The program also collaborates with other countries for cross-border coordination and to help them establish their own civilian stockpiles. Improvements were needed in collaboration when responsibility for the Stockpile was temporarily transferred to DHS.

Evidence: As part of the collaboration with State and local agencies, the program conducts exercises and developed a detailed guide to assist these partners in receiving, distributing and dispensing materiel from the Stockpile. CDC established an intragovernmental committee for the composition of the Stockpile with representation from DoD, other HHS entities, and offices within CDC. The program developed a State assessment tool to evaluate State readiness to receive, distribute and dispense Stockpile assets in an emergency. The program has a memorandum of agreement with the VA's National Acquisition Center for the purchase, storage, quality control, maintenance, and contingent deployment of medical materiel. The program works with DoD and FDA on the Shelf Life Extension Program (SLEP). The program maintains an agreement with the USAMMA to share in request and distribution responsibilities for anthrax vaccine adsorbed and smallpox vaccine and, for example, contracted with DSCP for auto-injectors. See also GAO-01-915, GAO-02-235T.

Does the program use strong financial management practices?

Explanation: CDC recently underwent a major effort to bring on the new Unified Financial Management System that is intended to reconcile any remaining weaknesses in this area. The system and CDC's associated process changes are to provide more real time data, streamlined processes, absolute funds control, and improved monitoring. UFMS automated funds control has enabled tracking of commitments, including aged balances, to provide better management information; improved financial planning using commitment data; produced tracking of current data on obligations to date, commitments and balances to provide information on spending actions, trends, plans, and the resulting impact on remaining funds availability; and provided historical data on all commitments, undelivered orders, payables, and payment transactions. Continued success will require positive documentation from independent auditors that indicates the new system has resolved weaknesses. Prior Performance and Accountability Reports noted continued weaknesses with CDC's financial systems, including a material weakness.

Evidence: Evidence includes error reports and other preliminary financial controls data from CDC's initial experience with fully implementing the Unified Financial Management System. CDC also automated reimbursable billings, enhanced year end closing transactions, implemented a new indirect cost methodology, and addressed staff needs. A December 2003 report by the OIG noted the agency had not implemented a system to allocate indirect costs until FY 2003, but found the new system to be a significant improvement for equity and accuracy. The OIG recommends CDC periodically review indirect costing methods. CDC has received five consecutive unqualified opinions. CDC issued 64 duplicate or erroneous payments in FY 2002, or 0.042 percent of all payments and has a 97 percent compliance rate for prompt payments. In November 2000, GAO (GAO-01-40) reported the agency's financial management capacity systems and procedures were insufficiently developed to address the agency's mission and budget growth.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The progam has identified and taken steps to correct management deficiencies through regular program reviews and audits and follow up actions. The program has improved its organizational structure in response to reviews from LMI and SAIC. The program has improved its site selection process and cGMP compliance program, retooled its State assessment system to improve the consistency and objectivity of ratings and is contracting with RAND to further improve the rating system to make it a more quantifiable measuring device. In other areas, the agency is extending the incorporation of performance measures into employee evaluations, setting priorities according to performance goals, and working to improve business practices. Management deficiencies highlighted in this section include insufficient use of clearly defined deliverables, performance characteristics and appropriate and credible cost and schedule goals. The program is becoming more specific with required performance characteristics early in the process with BioShield procurements.

Evidence: CDC implemented UFMS in April of 2005. The program uses a resource planning plan of action and milestones process to track progress on high-priority issues. Specific audits that informed corrections to management deficiencies include the LMI program management assessment, LMI assessment of the storage and distribution network, SAIC program and repository audits and cGMP compliance audits. The program has had increased input on the performance objectives of BioShield procurements.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: The program does not have a routine process for defining deliverables, performance characteristics and goals for procurements and does not routinely conduct an analysis of alternatives in advance of purchase. Vendor behavior is not the primary issue here. The program does consistently consider the suitability of products for emergency response use. The CHEMPACK project and Stockpile Resource Planning software are two cases where with a high level of complexity and sensitivity the program has managed procurements with clearly defined deliverables, performance characteristics and goals. The program's Intragovernmental Committee for Composition of the Stockpile met twice in 2004 to set a framework to help identify the most effective materiel to include in the Stockpile.

Evidence: Procurement contracts are fixed price with schedules for deliverables. Evidence for CHEMPACK includes LMI reports and CHEMPACK briefing materials and for the Stockpile Resource Planning software action and milestones.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program developed new long-term outcome measures that focus on treatments/prophylaxis for the appropriate response to known terrorist threats or public health emergencies for chemical, biological, radiological and nuclear threats and on successful annual exercises and number of active response units for delivering assets in response to multiple events within 12 hours. A large extent reflects considerable progress on one of the two measures and a one-year baseline and operational evidence for the second measure.

Evidence: By the end of 2005, the Stockpile prophylaxis capability will reach approximately 30 million people against anthrax, which in turn covers plague and tuleremia. By 2008, the program is to increase and sustain prophylaxis capacity to 60 million people for these threats. For the second measure, the program has conducted successful exercises that indicate the program can deliver assets to multiple locations within 12 hours. The program has also successfully responded to multiple events during the 2001 anthrax attacks with TARU delivery to Florida and North Carolina. The program delivered nearly 3.75 million tablets of three different antibiotics for post-exposure prophylaxis of employees in affected buildings, postal workers, mail handlers, and postal patrons. The program responded to 65 separate requests from 10 different States and the District of Columbia with an average response time from request to delivery of five hours.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program developed new annual performance measures, including an efficiency measure. A small extent reflects limited data on three of four annual measures, including the efficiency measure, with a baseline showing performance toward meeting targets.

Evidence: The program's proposed exercise activities will move beyond targeted exercises of the program and its partners to an annual exercise to test effective response to multiple events within the 12 hour response time. The program is establishing additional TARU units for response capability to multiple events. The program does not yet have data on the number of inventory discrepancies, but has begun tracking these data through the Stockpile resource planning system. The program's baseline for savings through the SLEP is $22 per dollar invested.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program does not have data to show progress on the newly adopted efficiency measure, but a small extent is given due to other cost savings the program has captured through its procurement practices. The program's purchase agreement with VA saved approximately $70 million on purchases of $309 million with a savings range from five percent to 74 percent and an average of 23 percent compared to Federal Supply Schedule prices. The program adopted a inventory management system that allows managers at storage sites to input receipts and inventory adjustments directly and quickly, replacing a process of paper submittals that would then be entered separately.

Evidence: CDC entered into a memorandum of agreement for the Shelf-Life Extension Program (SLEP) in October 2002. The program entered into an agreement with the Defense Supply Center Philadelphia that enables the program to access information management applications. The program is staffing intensive with 52 Federal full time equivalent employees and 177 contractors. In 2001, GAO (GAO-01-666T) credited the program with developing the vendor managed inventory approach that allows certain expiring stock for which there is a sufficient market to be rotated for full or partial credit to the seller.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: A not applicable rating is given because no comparable Federal, State, local or private sector programs exist. VA and DoD manage stockpiles for their clinical populations, but there are no comparable programs designed to protected the entire US population with a wide range of materials outside a defined clinical setting.

Evidence: VA and DoD stockpiles are small by comparison in funding and supply levels. Some industries and local hospitals maintain small caches of pharmaceuticals to protect their clients, but there is no comparable program in terms of scope, purpose, goals and delivery.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A rating of large extent reflects a relatively high level of scrutiny into the program from OIG and GAO that taken together indicate the program has improved its performance from when it was first established. The GAO, OIG and contracted reviews have identified areas of improvement, including the accuracy and accounting of inventory, and in many cases highlighted strengths of the program. The most recent reviews document progress the program has made in correcting management and oversight deficiencies and reflect improved performance. The LMI review of the push packages found the program has broad coverage of the US population, can respond equally well if a push package is taken out of circulation, and can withstand regional catastrophes. The system is less able to meet these objectives under a more aggressive response time. The review found areas of needed improvement with the program's storage and distribution network in each of the key areas, including coverage, redundancy and resiliency.

Evidence: A GAO review of a contested smallpox vaccine procurement (B-286167) denied the protest on the basis that the program had not acted improperly. A GAO review in 2003 (GAO-04-152) noted the effective contributions of the Stockpile in the anthrax attacks. A GAO review in 2000 (GAO/T-HEHS/AIMD-00-59) and again in 2001 (GAO 01-463) found the program was correcting weaknesses identified in an October 1999 review (GAO/HEHS/AIMD-00-36), but that further improvements were needed in risk assessments. A GAO review in 2001 (GAO-01-915) described the program's assets, capabilities and activities. The OIG review (A-04-02-08002) cited problems in oversight and procurement. An LMI assessment of CHEMPACK concluded positive results overall on the process and response from partners. LMI's management review found weaknesses with the identification of program requirements, the work breakdown structure, and the process for evaluating whether strategic objectives would be satisified and found the program did not identify risks to cost, schedule or performance in meeting the strategic objectives.

Were program goals achieved within budgeted costs and established schedules?

Explanation: A large extent is given because the program is unable to provide budget projections for ongoing costs and future procurements, but once a spend plan is enacted, the program has acquired and maintained assets within budget for scheduled purchases. Operation costs for management-in-use have been variable and slowly estimated. Schedules for each purchase are affected by market availability and manufacturing capability. While unenforceable, the program includes schedules in the purchase contracts through the VA National Acquisition Center. The program relies on VA to negotiate prices and terms of procurements and keep the program staff informed of delays that would affect the obligation of funds in the procurement of products or services.

Evidence: Evidence includes draft and final spend plans for prior years and the memorandum of agreement between VA and the program. Contracts are often sole source due to the narrow market for some materials and the program does not penalize contractors for delays. The program has not experienced significant cost growth associated with failure to meet project budgets or significant schedule delays post procurement.