Detailed Information on the
National Nuclear Security Administration: Nonproliferation and Verification Research and Development Assessment

Ensuring that new performance measures for next-generation nuclear detection technologies are tied to documented R&D goals, operational expectations, technical milestones, and decision/end-points.

Developing an activity prioritization process to guide budget requests and funding decisions.

Measure: Cumulative percentage of active research projects for which an independent R&D merit review of the project's scientific quality and mission relevance has been completed during the second year of effort (And again within each subsequent three year period for those projects found to be of merit).

Explanation:

Measure: Cumulative percentage of progress toward demonstrating the next generation of technologies and methods to detect Plutonium Production activities. (Progress is measured against the baseline criteria and milestones published in the "FY2006 R&D Requirements Document")

Explanation:

Measure: Annual index that summarizes the status of all NNSA nuclear detonation detection R&D deliveries that improve the nation's ability to detect nuclear detonations.

Explanation: Annually achieve timely delivery of NNSA nuclear explosion monitoring products. 90% target reflects good on-time delivery. Index considers factors beyond NNSA's control and impact on customer schedules.

Measure: Cumulative percentage of progress toward demonstrating the next generation of technologies and methods to detect Special Nuclear Material movement. (Progress is measured against the baseline criteria and milestones published in the "FY2006 R&D Requirements Document")

Explanation:

Measure: Cumulative percentage of progress toward demonstrating the next generation of technologies and methods to detect Uranium-235 production activities. (Progress is measured against the baseline criteria and milestones published in the "FY2006 R&D Requirements Document")

Explanation:

Measure: Annual number of articles published in merit reviewed professional journals/forums representing leadership in advancing science and technology knowledge

Explanation:Maintain scientific underpinnings of advanced R&D program.

Is the program purpose clear?

Explanation: The program has a clear purpose: to develop new technologies to improve U.S. capabilities to detect and monitor nuclear weapons production, proliferation, and testing worldwide.

Evidence: FY2006 Congressional Budget Request; NNSA Strategic Plan (DOE/NA-0010), pg. 35 (November 2004); U.S. Nuclear Detection System (USNDS) National Review; NA-22 Proliferation Detection (PD) Strategic Plan (January 2005) and NA-22 Nuclear Explosion Monitoring (NEM) Strategic Plan (September 2004); NA-22 FY2006 Program Guidance and Call For Proposals

Does the program address a specific and existing problem, interest, or need?

Explanation: The program is focused on a clearly identified critical need: The President has stated that "The United States has a critical need for cutting-edge technology that can quickly and effectively detect, analyze, facilitate interdiction of, defend against, defeat, and mitigate the consequences of weapons of mass destruction. It is widely recognized that increasingly sophisticated technologies are needed to detect nuclear proliferation." We are currently engaged in essential research and development to support the overall strategy against WMD proliferation. This includes: proliferation detection and nuclear explosion monitoring, our two primary mission areas. In addition, as stated by President George W. Bush, "The greatest threat before humanity today is the possibility of a secret and sudden attack with chemical or biological or radiological or nuclear weapons...America, and the entire civilized world, will face this threat for decades to come."

Evidence: National Strategy to Combat Weapons Of Mass Destruction (December 2002); NNSA Strategic Plan; NA-22 PD&NEM Strategic Plans; GAO Report 02-904, "Nonproliferation R&D: NNSA's Program Develops Successful Technologies, but Project Management Can Be Strengthened" (August 2002); classified interagency documents.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program is designed to avoid redundancy and duplication with other organizations as demonstrated by a long history of collaboration, information-sharing, and openness in cooperating with our interagency partners in a wide variety of interagency forums. Several interagency working groups have been specifically chartered to ensure that redundancy and duplication are not present: Space Technology Alliance (STA), Nonproliferation and Arms Control Technology Working Group (NPAC-TWG) (co-chair with DoD and State), Counterproliferation Program Review Committee (CPRC) (DoD, DOE (vice-chair), and Intelligence Community). The NPAC-TWG and CPRC have goals for eliminating redundant or duplicative research activities. NA-22 is part of the interagency support for DNDO and anticipates detailing four (4) scientists to staff the DNDO organization. The program is unique because it is the only DOE office tasked with creating new technologies for detecting nuclear weapons development. Under the Atomic Energy Act of 1954 (as amended) the Department of Energy is specifically identified as the sole governmental agency to control the dissemination of information and technology related to nuclear weapons and special nuclear material (SNM). This access, expertise, and historical perspective has resulted in the recognition of this organization as the focal point for long-term nonproliferation research and development. This office is an active participant in the operation and planning for the newly created Domestic Nuclear Detection Office (DNDO). The Program uses interagency representatives for proposal evaluation and project independent review.

Evidence: Atomic Energy Act of 1954 (as amended); NNSA Strategic Plan; Detailed Justification for Program Budget 2002; GAO Report GAO-05-157, "Weapons of Mass Destruction: Nonproliferation Programs Need Better Integration" (January 2005); NA-22 annual program reviews, project reviews, and peer reviews. The Program is an active participant in the following interagency working groups: STA, NPAC-TWG (co-chair with DoD and State), CPRC (DoD, DOE (vice-chair), and Intelligence Community).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is free of major flaws that would limit effectiveness or efficiency. Current planning and restructured activities in the program have greatly increased program effectiveness and efficiency. Following the transfer of major elements of this program to DHS in 2002, the office has efficiently reorganized and established the necessary structure to refocus on the core nuclear nonproliferation mission. As recognized in OMB Report 04-653, "NNSA has improved the controls it uses to provide the operational and finanacial data that its managers use to monitor project costs, milestones, and deliverables. The program has implemented a Web-based project management information system that is designed to contain the information needed for proposal evaluation, budget planning, funding decisions, preparations of work authorizations, responses to congressional inquiries, and other reports on the status and costs of R&D projects managed by the program. In addition, the program has emphasized the importance of perfomance metrics and the use of the OMB PART."

Evidence: FY06 Program Guidance and Call for Proposals; WebPMIS Requirements Document; Draft NNSA Business Operating Plan 4.1; FY06 BAA coordination documentation; GAO Report 04-653, "DHS Needs a Strategy to Use DOE's Laboratories for Research on Nuclear, Biological, and Chemical Detection and Response Technologies" (May 2004), Appendix 1: "NNSA Has Strengthened Its Project Management Controls".

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program is effectively targeted and directly involves end-users from project selection through demonstration processes. Funding priorities are based on NNSA Strategic Planning Guidance, the National Strategy to Combat Weapons of Mass Destruction, and our risk-informed decision-making.

Evidence: GAO Report 04-653, Appendix 1; NNSA Strategic Planning Guidance (2005-2009); National Strategy to Combat Weapons of Mass Destruction; classified endorsement letters.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program uses specific long-term (outcome based) measures to meaningfully assess progress in achieving its program purpose. They are 1) by 2016, demonstrate the next generation of technologies and methods to detect Uranium-235 Enrichment activities, 2) by 2015, demonstrate the next generation of technologies and methods to detect Plutonium Reprocessing activities, 3) by 2013, demonstrate the next generation of technologies and methods to detect Special Nuclear Material movement, and 4) annually achieve timely delivery of NNSA nuclear explosion monitoring products per user requirements to sustain and improve the nation's ability to detect nuclear explosions. 5) Ensure that 100% of the active research projects have completed an independent R&D peer assessment of the projects' scientific quality and mission relevance during the second year of effort and again within each subsequent three year period. 6) annually achieve goal of 200 articles published in peer reviewed professional journals/forums representing leadership in advancing science and technology knowledge.

Evidence: NA-22 Proliferation Detection (PD) and Nuclear Explosion Monitoring (NEM) Strategic Plans; Draft FY06 NA-22 Performance Measures; NA-22 FY2005 and FY2006 Program Guidance and Call for Proposals; NA-22 FY2006 Budget Request; NNSA PPBE Process; Prior focused planning efforts to define long-term goals include: "Improving Laboratory Tools and Analytical Capability to Deter Nuclear Proliferation" (2001), "Active Techniques for Detecting Shielded HEU" (2001), "Confidence Building for International Safeguards" (2001), "Long-range Detection of Special Nuclear Material" (2001), and "Signatures and Observables" (2001).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has ambitious targets and timeframes through meaningful long-term applied research programs, which by definition are focused on difficult problems. A portion of the R&D program is inherently risky. Program risk is mitigate by technical criteria, peer reviews and management controls that ensure the ambitious long-term goals are realistic and achieveable. Based upon programmatic assessment of the state-of-the-art of science and technology relating to the aforementioned measures the program is confident that these targets and timeframes are ambitious and the program is in the process of externally validating these assessments.

Evidence: Draft FY06 NA-22 Performance Measures; NA-22 PD&NEM Strategic Plans; NA-22 FY06 Program Guidance and Call for Proposals; NA-22 FY06 Budget Request; NNSA PPBE process.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has 6 annual measures that meaningfully demonstrate progress toward achieving long-term measures. In FY07, 1) 15% completion of uranium enrichment detection roadmap/program plan; 2) 20% cumulative percentage towards completion of plutonium reprocessing detection; 3) 20% cumulative percentage towards completion of special nuclear material detection technologies; 4) Achieve an annual index of 90% timely delivery of NNSA nuclear explosion monitoring products per user requirements to sustain and improve the nation's ability to detect nuclear explosions (Both ground and space nuclear explosion monitoring sub-programs have decades-long successful history of meeting requirements of the US Atomic Energy Detection System). 5) Achieve annual target of 100% for which an independent R&D peer assessment of an active project's scientific quality and mission relevance has been completed during the second year of effort . 6) Achieve annual target of 200 papers published in peer reviewed journals.

Evidence: NA-22 PD&NEM Strategic Plans; Draft FY06 NA-22 Performance Measures; NA-22 FY2006 Budget Request; NNSA PPBE process; classified interagency documents.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Please see measures section for detailed information regarding baselines and targets for our annual measures. The outcomes are measuresd against a 7-step R&D process which are 1) Set goals objectives and requirements 2) Roadmap\program plan 3) Technical feasibility studies 4) External exptert\user reviews 5) Ranking or alternative approaches and down-selection 6) Research phase on selected approaches 7) Demonstration of developed technolgies and methods

Evidence: NA-22 FY2006 Program Guidance; FY2006 Congressional Budget Request; FYNSP; Draft FY06 NA-22 Performance Measures, NA-22 NEM Strategic Plan, classified interagency documents., narraative explanation of milestones.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: "The partners are required to work toward the program's long-term goals before and during the execution of the projects. Long-term goals are translated into Statements-of-Work and Lifecycle Plans within the program's custom project management system WebPMIS, which become work authorizations for executing resources at the National Laboratories, universities, and industry. As required in the program guidance document, proposals from all partners are required to identify the purpose of intended research in the context of the program's goals. "

Evidence: NA-22 FY2006 Program Guidance; WebPMIS Requirements Document; Interagency MOUs with Partnering Agencies (USNDS MOA and Classified MOAs); FY2005 and FY2006 Program Guidance and Call for Proposals; Annual portfolio reviews (includes potential users, principal investigators, NA-22 management, and representatives of other interested government agencies used to ensure that all partners understand and commit to working toward a common set of goals).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program conducts a wide mix of program reviews with nonproliferation monitoring technology providers and customers. Annual and long-term performance goals are communicated to partners through annual reviews and program strategic plans. Users and developers maintain standing independent review groups to conduct or participate in program reviews assessing new technologies that have reached a level of maturity for operational systems. The program has been previously examined by GAO in 2002 and 2004 and by an externatl advisory panal in 2000.

Evidence: Annual Seismic Review Panel; Annual Satellite Systems Review Panel; Program and Project Merit Peer Reviews; Annual Program Review by NNSA; Proliferation Detection Program Annual Technical Information Exchange conference; Portfolio Review of Trace Effluent Detection Analysis. GAO Report 04-653, GAO Report 02-904, report by the Nonproliferation and National Security Advisory Committee (February 25, 2000), JASON report on LIDAR: number JSR-93-310, September 1994

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: NNSA budget requests are explicitly tied to anticipated annual and long-term performance goals via the comprehensive Planning, Programming, Budgeting and Evaluation (PPBE) process. Long-term performance goals established/validated during the Planning Phase are linked in a performance cascade to annual targets and detailed technical milestones. During the Programming Phase, budget and resources trade-offs and decisions are evaluated based on the impact to annual and long-term performance measures. These NNSA performance-planning-budgeting decisions are documented in the Program Decision Memorandum (PDM) and used to develop the budget requests during the Budgeting Phase. Program and financial performance for each measure is corporately monitored and assessed during the Execution and Evaluation Phase.

Evidence: NNSA PPBE Guidance Documents; Annual Congressional Budget Request; Annual Program Decision Memorandum,and FY05-09 FYNSP; JOULE Quarterly reports

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has devoted significant efforts to correct strategic planning deficiencies. Following the transfer of major elements of this program to DHS in 2002, the office has efficiently reorganized and established the necessary structure to refocus on the core nuclear nonproliferation mission. The NEM Strategic Plan was updated in 2004 and a vision 2015 study is underway. A Proliferation Detection Strategic Plan was developed in 2005. Our program planning process is guided by NNSA the PPBE process.

Evidence: National Security Strategy; National Strategy to Combat Weapons of Mass Destruction; NNSA Strategic Plan; DOE Strategic Plan; GAO Report 04-653, Appendix 1; NA-22 PD&NEM Strategic Plans; "Suggestions for Strategic Planning for the Office of Nonproliferation Research and Engineering", Rand Report (2002) for the Nonproliferation and National Security Advisory Committee.

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: The program conducts external project reviews assess and compare the approach and benefits of each project with other known R&D efforts. Every effort has been made to work with other organizations to ensure program efforts with similar goals are coordinated through a wide variety of interagency working groups.

Evidence: NPAC-TWG; CPRC, Intelligence Community Advanced Research and Development Committee (ARDC); Space Technology Alliance; Satellite Systems Review Panel; Seismic Review Panel; classified MOUs; UK-MOD, DOE, DHS MOA (2004).

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: Program priorities are evaluated through the NNSA PPBE process to meet mission requirements and technology needs in partnership with user agencies. NA-22 has rated this category as "No" because of a lack of evidence of a documented process.

Evidence: USG Treaty Monitoring Requirements, MOUs with partnering agencies, coordination with partnering agencies on monitoring mission needs, NNSA PPBE process. NPAC-TWG; CPRC; ARDC; Space Technology Alliance; Seismic Review Panel; Satellite Systems Review Panel; classified MOUs.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program regularly collects timely and credible performance information. Performance is reviewed quarterly and annually to correct systematic project management deficiencies. The program receives monthly financial reporting from each project. This information is used to determine whether corrective action is necessary. Federal managers review official DOE accounting data monthly as a check on contractor reports.

Evidence: Quarterly Project Reports; Independent Project Reviews; Annual Seismic Review Panel; Annual Satellite Systems Review Panel; WebPMIS Requirements Document; Monthly Financial Reports

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal program and project managers are responsible and accountable for the specific goals of the programs becasuse there are elements in there performance appraisals that are linked to the NNSA Strategic Plan and goals. The National Laboratories and other performers are held to project milestones and deliverables. Their prior performance is one of the elements used by the program to assess future funding for proposals.

Evidence: DOE Performance and Appraisal system, DOE Performance and Accountability Reports, National Laboratory Annual Assessments., Project Quarterly reports and lifecycle plans

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Programs funds are obligated in a timely manner and historically about 75% obligated by the end of the first fiscal quarter and 100% obligated by the end of the fiscal year. Program funds are clearly identified for individual projects through work authorizations when they are obligated. Strict project selection and review procedures, as well as rigorous implementation oversight, ensure that project funds are used in accordance with program objectives. In recent years, continuing resolutions have created challenges to early obligation of funds. It is estimated that the program's uncosted estimate will be $25.1M or 8.1% or our adjusted budget by the end of the fiscal year. This is well below the DOE CRO threshold of 12-14%.

Evidence: Monthly Financial Reports, DOE Financial Management System, DOE Financial Plan, Monthly Authorized Funding Plan., FY05 Project Uncostd Balances Document

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program is assigning itself a "No" in this area because of the lack of specific written procedures to measure the efficiencies. All contracts, financial assistance, and grants are sent through the NNSA service center which implements all appropriate regulations and policies appropriate to each vehicle.

Evidence: PPBE process, Contractual Measurement by NA-60, Broad Agency Announcements and University Research Call in FY2004.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program performs routine coordination with major partners including DoD, DHS, DOS, DOJ, and Intelligence Community elements. Coordination with US agencies is formalized in MOUs and MOAs, as needed. Formal interagency coordination mechanisms include: Nonproliferation and Arms Control Technology Working Group (co-chaired by DOE, DoD, and DOS), Counterproliferation Program Review Committee (DoD, DOE (vice-chair), and Intelligence Community), Space Technology Alliance, Intelligence Community Advanced Research and Development Committee; Program and project reviews include interagency participation.

Evidence: USNDS MOA; classified MOAs; NPAC-TWG Annual Report, CPRC Annual Report; Space Technology Alliance; Intelligence Community Advanced Research and Development Committee, Proliferation Detection Technical Information Exchange Meeting (April 11-15, 2005), Trace Effluents Detection and Analysis Portfolio Review (March 25, 2005).

Does the program use strong financial management practices?

Explanation: The program uses strong financial management practices outlined in DOE's financial management policies, procedures, and practices that meet all statutory requirements. DOE provides accounting services to the program and they are free of any material internal control weaknesses. The DOE's financial statements have been given a clean audit opinion in 8 of the last 9 years. The program's managers' review accounting reports monthly to monitor obligations and costs for all projects and sites.

Evidence: DOE Financial Management Orders, Monthly NNSA Financial Reports, NNSA PPBE Guidance Documents located on the NNSA web-site; NA-22 WebPMIS system

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Meaningful steps have been taken by the program during the last two years, including instituting independent peer reviews, updating or completing program element strategic plans, and developing outcome-based performance metrics. The program has hired additional staff and continues to regularly self-review the program to identify areas for improvement.

Evidence: List of Independent Peer-Reviewed Projects (April 25, 2005); NA-22 PD&NEM Strategic Plans; FY 2007 Program Metrics; Improved Project Management System (WebPMIS Documentation System); GAO Report 04-653, Appendix 1: "NNSA Has Strengthened Its Project Management Controls".

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: The program plans, allocates, and tracks funds using the internal Web-based Project Management Information System (WebPMIS) and maintains program quality through independent reviews. Independent reviews provide assments to program managers concerning project risk, potential alternative approaches, and complimentary research efforts.

Evidence: GAO Report 04-653, Appendix 1; Draft WebPMIS Requirements Documentation; and documentation of external reviews of FY2005 projects.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: "Historically, the program delivers nonproliferation monitoring products and services on schedule and in accordance with requirements and specifications. Some examples include 1) 40 years of delivering nuclear explosion monitoring instruments without failure; 2) 10-year remote spectroscopy program for national security applications; 3) three research satellites developed and launched over last 15 years exceeding mission performance requirements; 4) chemical and biological agent detection program, transferred to DHS, including the only effective biological detection system for protecting civilian populations post 9/11. Our measurement methods have recently been significanlty revised to to be process orientated rather than output based. We are curently on track to demonstrate: 1) the next generation of technologies and methods to detect Uranium-235 Enrichment by 2016 2) the next generation of technologies and methods to detect Plutonium Processing by 2015 and 3) the next generation of technologies and methods to detect Special Nuclear Material movement by 2013. We are also on track for: 4) timely delivery of NNSA nuclear explosion monitoring products per current user to sustain and improve the nation's ability to detect nuclear explosions 5) Ensuring that 100% of the active research projects have completed an independent R&D peer assessment and 6) Achieve annual target of 200 papers published in peer reviewed journals. "

Evidence: DOE Performance and Accountability Reports (2001-2004); GAO-02-904; DOE Research and Technology Against the Threat of Weapons of Mass Destruction, report by the Nonproliferation and National Security Advisory Committee (February 25, 2000); R&D100 awards; classified endorsement letters from user agencies; classified and unclassified program documentation.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has consistently achieved its annual performance targets. Throughout the year, NA-22 program and project managers rate and document performance in terms of annual performance goals. Goal achievement is documented in the DOE Performance Accountability Reports: In FY03 achieved 4 of 4, in FY04 achieved 3 of 4, in FY05 achieved 3 of 4.

Evidence: DOE Performance and Accountability Reports (2001-2004)

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has been flat-funded over the last decade. Over time, efficiency has improved in the program through the higher quality of its nonproliferation monitoring products achieved at the same annual cost. The program has successfully restructured its resources and core competencies following the FY 2003 transfer of approximately one-third of its R&D program to the Department of Homeland Security. The program is assigning itself a "Small Extent" in this area because of the lack of specific written procedures to measure the efficiencies.

Evidence: "Briefing to OVP on program funding; the streamlining of the program from three subprograms to two; implementation of the webPMIS; 2003 and the Homeland Security Act. "

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The program compares favorably to other R&D programs. User agencies have come to depend on the program for long-term R&D in support of their missions. External reviews of the R&D Program consistently rate the quality of the research program high. A large portion of the NA-22 portfolio represents unique, long-term nonproliferation R&D.

Evidence: DOE Research and Technology Against the Threat of Weapons of Mass Destruction: February 25, 2000; GAO Report 04-653; GAO Report 02-904.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Multiple independent evaluations of program elements indicate effectiveness in meeting user needs. Users from federal, state and local agencies that use technology developed by the NNSA Nonproliferation R&D program have found the technology useful and remarked on the effective relationship with the Program. The NEM program element recently underwent a national review that generally supported the program results/quality and suggested a slight redirection of priorities due to changing world circumstances. A large percentage of the Department of Homeland Security's Science & Technology successes reported to date can be traced back to the NNSA Nonproliferation R&D Program accomplishments of prior years.

Evidence: Annual Seismic Review Panel; Annual Satellite Systems Review Panel; Program and Project Merit Peer Reviews; Annual Program Review by NNSA; Proliferation Detection Program Annual Technical Information Exchange conference; Portfolio Review of Trace Effluent Detection Analysis; DOE Research and Technology Against the Threat of Weapons of Mass Destruction, report by the Nonproliferation and National Security Advisory Committee (February 25, 2000); GAO Report 02-904; 2003 USNDS National Review.