Detailed Information on the
National Nuclear Security Administration: Nuclear Weapons Incident Response Program Assessment

Investigating impediments to the performance elements that comprise the program's ability to respond to and mitigate nuclear and radiological incidents worldwide.

Improving coordination of priorities and initiatives across NNSA sites and continually assessing emergency response to help program managers identify and fix deficiencies within program elements.

Measure: Emergency Operations Readiness Index measures the overall organizational readiness to respond to and mitigate radiological or nuclear incidents worldwide. (This Index is measured from 1 to 100 with higher numbers meaning better readiness--the first three quarters will be expressed as the readiness at those given points in time where as the year end will be expressed as the average readiness for the year's four quarters).

Explanation:The program has a single long-term outcome measure to meaningfully assess the program's ability to achieve its purpose. This index is a holistic measure of the program's ability to respond to and mitigate nuclear or radiological incidents worldwide. The index measures several key factors for achieving readiness including personnel staffing levels, personnel training, equipment levels, equipment condition, number of successful no-notice exercises and the dependability of the emergency communications and notification network.

Is the program purpose clear?

Explanation: The Nuclear Weapons Incident Response Program purpose is clear - respond to and mitigate nuclear and radiological incidents worldwide. This purpose includes: (1) developing plans to minimize the impact of emergencies on workers, the public, and the environment; (2) responding to the emergency events; and, (3) providing guidance to NNSA facilities on what to do if an emergency happens.

Evidence: The assemblage of policy notes that provide operational guidance; DOE Order 151.1C; the NA-40 strategic plan which is in a cascade from the DOE and NNSA strategic plans, and a variety of guidance memoranda on specific topics.

Does the program address a specific and existing problem, interest, or need?

Explanation: The Nuclear Weapons Incident Response (NWIR) Program is the Nation's unique resource to address the ever-present threat of nuclear/radiological terrorism through direct, tailored emergency response or consequence management actions. NWIR ensures that the right people, equipment and training are readily in place and available to respond to and mitigate nuclear and radiological incidents worldwide. The US Government has confirmed a credible and growing threat of attack against the US homeland. In addition, natural emergencies and accidents sometimes occur, requiring immediate response, coordination and mitigation from NWIR expertise, assets and facilities.

Evidence: DNFSB Technical Report 21 addresses the viability of the emergency management program. PDD 39 & 62 outline DOE radiological roles. Interagency responsibilities are delineated in the National Response Plan and Radiological/Nuclear Incident Annex.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program is a unique national resource and unique federal entity within the US Government. First, it provides focused emergency management expertise throughout the United States. Secondly, it provides the highly specialized technical expertise for addressing the threat from nuclear/radiological terrorism that represents a core capability of the Department and the Nation.

Evidence: The National Response Plan and the Radiological/Nuclear Incident Annex clearly designate the DOE's roles and responsibilities.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program design is free of major flaws and is organized to provide the most effective and efficient approach in addressing potential nuclear/radiological incidents. Documents/plans related to optimizing the program's organizational structure and associated operations have been developed and disseminated on a continuing basis for implementaton. A solid program concept has been developed and approved by DOE and NNSA. All activities and the schedule for their accomplishment are implemented through an integrated plan whose milestones and deliverables are monitored through monthly reviews and various reporting mechanisms.

Evidence: Progam implementation documents/ plans and policy notes; the President's Budget/NNSA FYNSP; NNSA Planning, Programming, Budgeting, and Evaluation process input.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program's resources are effectively targeted to reach the intended beneficiaries. The basic structure of the emergency management program has been in place since 1991. Areas of improvements or focus are periodically incorporated into revised DOE Orders and NA-40 policy notes. The emergency response program has evolved over the years to address changing focus and national requirements. For example, the program has developed an enhanced regional response capability that can respond to dirty bombs or other terrorist devices. The ongoing regionalization of the Radiological Assistance Program (RAP) teams has decreased response times. In addition, the program funds are clearly targeted through specific work plans for the nuclear/radiological assets that fulfull the program's purpose.

Evidence: Baseline organizational structure as outlined in the Realignment of the Office of Emergency Management functions memo approved by the Secretary on June 17, 2002, and the NNSA emergency management authorization memo signed by the Administrator on September 26, 2002. Participation in the NNSA business drivers survey of current and future workforce needs.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has a single long-term outcome measure - readiness-- to meaningfully assess the program's ability to achieve its purpose. This measure encompasses the "ready" status of personnel, training of personnel and availability and accessibility of equipment and condition needed to respond to any nuclear or radiological incidents worldwide. This long-term measure is to achieve and maintain an Emergency Operational Readiness Index of 91 or higher. The Readiness index is a holistic measure of the program's ability to respond to and mitigate nuclear or radiological incidents worldwide. While single in nature, the index actually measures quantitatively several key factors for achieving readiness including personnel staffing levels, personnel training, equipment levels, equipment condition, number of successful no-notice exercises and the dependability of the emergency communications and notification network. The Readiness index is strict -- the overall rating is the lowest rating of any component. This approach has been undertaken because we have to be ready to respond to an increasing threat of nuclear or radiological incidents. This measure assesses emergency response readiness and helps program managers identify and correct deficiencies within key elements of the program.

Evidence: Policy Note #10, ARMS implementation plan, NA-40 Strategic Plan; FY 2007 OMB Congressional budget request; FY 2006 JOULE.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Given the dramatic increase in program assistance post-9/11, these long-term measures are very ambitious. However, comprehensive reviews and planning show that although very challenging, targets are realistic and achievable. The Readiness index puts our focus on those items critically important to NWIR. It also allows us to better align our budget and measures and forces a focus on problem areas.

Evidence: Policy Note #10; NA-40 Strategic Plan; FY 2007 Congressional budget request; FY 2006 JOULE.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has annual targets to meaningfully assess the program's ability to achieve its long-term goal. These annual tragets began with the adoption of the Emergency Operational Readiness Index. The initial operational readiness index was 71 in FY 2005. We increased this to the long-term operational readiness goal of 91 in FY 2006.

Evidence: NA-40 Strategic Plan, Policy Note #10, the Single Performance Measure guidance document; FY 2007 Congressional budget document request; FY 2006 JOULE.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has a detailed and ambitious baseline in each annual target. Annual Emergency Operational Readiness Index targets are broken down into baselines for six areas: personnel staffing, personnel training, equipment levels, equipment condition, no-notice exercises and dependability of emergency communications. Each of these six areas is further divided into mission areas. For example, at the lowest level of the baseline, the program identifies and measures against the number of health physicists needed to work in a particular regional asset (such as a Radiological Assistance Program "RAP" team); or, the number of radiation detectors available needed for search operations. Since operational readiness is only as good as its weakest link, the index defaults to the lowest index level in the overall rating.

Evidence: NA-40 Strategic Plan; FY 2007 Congressional budget request; FY 2006 JOULE .

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Full field participation by federal and contractor senior personnel were utilized as part of the Strategic Plan Task Force to develop annual and long-term goals for the emergency operations programs. Field federal personnel also participated in the development of program execution plans (PEPs). We test our readiness to respond through no-notice exercises and externally generated interagency exercises; e,g, FORWARD CHALLENGE.

Evidence: The President's Budget/NNSA FYNSP; M&O Contractor PEP's and evaluations; NA-40 Strategic Plan; Memorandum of Understanding between DOE/NNSA and DHS pursuant to the Homeland Security Act for that National incident Response Team (NIRT).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Office of Independent Oversight and Performance Assurance (OA), the Office of the Inspector General, and the General Accounting Office routinely audit components of the program.

Evidence: Audit reports from Office of Independent Oversight and Performance Assurance (OA).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Each NNSA program is managed using long term performance goals with annual targets that cascade seamlessly from the NNSA strategic plan, and also link to supporting milestones, deliverables and the budgets for program performers. NNSA's program performance measures are the framework for resource allocation decisions made in the annual Planning, Programming, Budgeting, and Evaluation (PPBE) processes. The Budget and Reporting (B&R) categories for the NWIR program are linked to their performance metrics. All direct and indirect costs to attain the performance results for the NWIR program are reported in the B&R categories.

Evidence: These NNSA performance-planning-budgeting decisions are documented in the DOE Program Decision Memorandum (PDM) and used to develop the budget requests during the Budgeting Phase. Program and financial performance for each measure is corporately monitored and assessed during the Execution and Evaluation Phase. The President's Budget/NNSA FYNSP; NNSA PPBE Guidance Documents located on the NNSA web-site; NNSA Program Decision Memoranda, and Administrator's Final Recommendation.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Strategic Planning, Program Reviews, and Field Program Reviews are used to develop strategic actions and budget plans for addressing planning deficiencies. For example, as a result of the readiness review, we identified problems with response times which necessitated making improvements in the Radiological Assistance Program (RAP) teams' operations. These changes have resulted in improved response times.

Evidence: NNSA PPBE Guidance Documents located on the NNSA web-site; FY2007 Congressional budget request, FY2007 DOE PDM, and FY2007 - 20011 FYNSP.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Emergency Management performance metrics are developed by field organizations and submitted through cognizant line management organizations quarterly. Emergency response asset team equipment and personnel readiness are substantiated through the Asset Readiness Management System (ARMS). Program Directors within NA-40 conduct a monthly review and report readiness status relating to the program to the Deputy Associate Administrator.

Evidence: The metrics contained in the ARMS data pertain to personnel, training, equipment and equipment condition.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and program partners are held accountable for cost, schedule and performance results. The DOE Work Authorization Statement (WAS) and associated Program Execution Plans (PEPs) and budget validation reviews are formal process used to quanlify and quantify work deliverables, monthly financial reports identify cost, program reviews and the JOULE Report identify schedule and performance results. Federal employee performance appraisals include performance elements that link to program performance measures.

Evidence: WAS and PEP documents, FY 2006 JOULE; Federal employee performance appraisals.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Formal DOE and NNSA processes monitor expenditure of funds based on budgetary allocations as approved through the Chief Financial Officer. Program managers use monthly financial reports along with the associated WAS and PEP to track and manage the obligation and costing of all program funds.

Evidence: DOE/NNSA monthly financial reports, PEP documents; AFP and WA files; monthly DOE financial data warehouse reports; year-end spending reports; and Federal audit clearinghouse.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has procedures for measuring and achieving efficiencies and cost effectiveness in program execution. For example, an internal review of the RAP program completed in December 2005, revealed that the RAP regions had carryover funds. At the same time, because there was a temporary need for a specialized scientist (who happened to be in RAP Region 0) for a special project at Headquarters, with an accompanying need for the funding to support the temporary transfer, the review process allowed us to reallocate some of the carry over funds to cover the additional expenses for bringing the expert into Headquarters for the special project.

Evidence: Program managers review financial data monthly and receive monthly or bi-monthly cost and performance reports from the field offices.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program provides emergency response support to seventeen federal agencies, including DoD, FBI and others. In addition, we collaborate effectively and on a regular basis with related State and Local programs involved in emergency response activities. This collaboration takes place through several key coordinating/working groups, and allows for the most efficient and effective implementation of key DOE/NNSA emergency response assets. Given the current operational tempo, this close coordination has been instrumental in ensuring the seamless integration of these assets into the overall response to potential nuclear/ radiological incidents. A good example would be the June 2005 Pinnacle interagency exercise, in which Program members played a key role in ensuring a successful outcome.

Evidence: Interagency planning documents and cooperation on real-world operations including Nationally Significant Secuity Events and interagency plannnig such as the National Response Plan and other planning documents between DOE/NNSA and the Department of Homeland Security, the Department of Defense, the Department of State, and the Federal Bureau of Investigation; Pinnacle, the TOPOFF exercise series, and other exercise documents.

Does the program use strong financial management practices?

Explanation: NWIR is covered by DOE's financial management policies, procedures, and practices that meet all statutory requirements. The accounting services for NNSA are provided by DOE and these are free of material internal control weaknesses. The DOE financial statements have been given a clear audit opinion in 8 of the last 11 years. The Budget and Program staff review cost expenditure reports on a monthly basis to monitor obligations and costs for all projects and sites.

Evidence: DOE's financial statements have been given a clear audit opinion in 7 of the last 8 years. DOE Financial Management Orders; NNSA PPBE documents on the NNSA PPBE web site; and DOE financial data warehouse reports.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: We conduct quarterly and annual program reviews which help to identify program management deficiencies. Identified deficiencies are tracked by the respective program manager until they are corrected. Management deficiencies identified by on-site reviews, DOE Office of Independent Oversight and Performance Assurance inspections, DOE Inspector General inspections, Defense Nuclear Facilities Safety Board recommendations and reports, and independent reviews are resolved through formal corrective action plans. For example, the annual review for FY 2005 assisted in management's making a determination that contractor growth was too high for the budget to support. We met with the program's support contractors and, through a series of cooperative meetings, achieved the elimination of costs sufficient to bring contractor growth back in line. The Readiness Index and its underlying metrics puts our focus on those items critically important to NWIR; Readiness and forces a focus on problem areas.

Evidence: Program review documents; audit reports; exercise after-action reports; quarterly readiness reviews and identified deficiencies and corrective actions.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has demonstrated progress in achieving its long-term goal to a large extent. The new Emergency Operations Readiness Index was initiated in FY 2005 with an index of 71. Through FY 2006, 2nd quarter, we have increased the Index to 74. We have identified a need for training new staff members that have recently been hired. When this training is completed by June 2006, we will increase the Index to the long-term goal of 91.

Evidence: JOULE FY06 data; ARMS data and reports.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program and its partners have achieved its annual performance targets to a large extent. During FY 2005, the program conducted seven no-notice exercises, responded to 113 emergency call-outs, resolved 100% of the call-outs, and completed the exercises successfully. Year-to-date in FY 2006, the program has responded to 60 emergency call-outs and resolved 100% of them.

Evidence: JOULE FY06 data; ARMS data and reports.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The new Emergency Operations Readiness index savings will be realized in FY 2007. Emergency Response has two Research and Development (R&D) programs. The Technology Integration Program works with our response assets to establish scientific and technical needs, conducts reviews of proven technologies to fill those needs, and develops tools in the form of hardware, software, or methods to be fielded by our teams. Within the program each project is subjected to a testing and evaluation process to ensure our tools meet expectations. This program has been very successful in exploring and developing new technologies for use by our deployment teams. For example, in FY 2005, the program funded in excess of seventy projects at eleven laboratories around the DOE complex. New technologies help us achieve greater efficiencies because the instruments result in either being faster or less expensive. This is because they provide greater accuracy in measurements as they are easier to use. As an example, the Mobile ECN enables more efficient and secure team deployments because the teams now take their communication gear with them. The purpose of Render Safe R&D program is to explore higher risk projects focused on developing new cutting edge science and technology necessary to successfully render safe a nuclear weapon or improvised nuclear device. This program, which was new in FY 2005, has already matured to the point where we are confident we can start manufacturing and testing first generation equipment by FY 2008. Both programs also work closely with other research organizations to help drive research efforts to support our needs in areas where current technology falls short and to help reduce duplication of efforts.

Evidence: JOULE FY 06 data; ARMS data and monthly cost reports.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Mission accomplishment by the program has been second to none. 100% of emergency response requests have been satisfied.

Evidence: All requested missions have been accomplished.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The Office of the Inspector General (OIG) recently completed a review of the program and validated the mission accomplishment of the Office of Emergency Operations. NA-40 is currently involved with two on-going program evaluations in FY 2006: the DOE internal audit of the Radiological Assistance Program and the General Accountability Office audit of the National Emergency Support Team (NEST). Results will be forthcoming. Internally, we evaluate each deployment and exercise for lessons-learned. The lessons-learned resulted in changes to policy notes that allow us to correct errors and prevent errors from occurring.

Evidence: OIG Review: The Department's Continuity Planning and Emergency Preparedness (DOE/IG-0657 August 2004).