Detailed Information on the
Natural Gas Technology Assessment

Refocus the program on longer-term, high-risk research that will provide public benefits that otherwise would not have occurred without the program's intervention.

Develop guidance that specifies a consistent framework for analyzing the costs and benefits of research and development investments, and use this information to guide budget decisions.

Work with the Congress to terminate the program this year.

Conduct periodic, independent evaluations of all program components in order to focus program activities and target resources toward achieving relevant and beneficial outcomes that otherwise would not have occurred without the program intervention.

Identify quantitative test criteria that will be used to validate all Current Year and Budget Year performance goals when test results become available.

Measure: Additional economically recoverable domestic gas resources (trillion cubic feet, Tcf).

Explanation:By 2015 develop technologies to expand the 2002 domestic gas economically recoverable resource base by 100Tcf. ($3.50/million cubic feet price, based on Annual Energy Outlook 2003 assumptions.) Annual performance will be measured by modeling, and assumes that 8 projects from current portfolio will successfully achieve critical milestones in each year, thereby contributing additional economically recoverable domestic gas resources. Technology products include prototype and/or field-tests of innovative means for resource characterization, exploration success rate, production cost reduction, and/or increased productivity. Critical technology areas include advanced drilling, stripper-well enhancement, and gas storage.

Measure: Technically recoverable resources of natural gas from methane hydrates (trillion cubic feet, Tcf).

Explanation:Without DOE R&D, initial production of natural gas from methane hydrates is unlikely until about 2025. With DOE R&D, initial production can be accelerated by about 5 years to 2020 (at the FY05 level-of-R&D-effort which is a reduction from the current FY03 effort). Based on estimated ratios of technically-recoverable resources (TRR) to production levels, the TRR in 2025 will be roughly 25 Tcf, approximately 20 Tcf greater than in the case without DOE R&D.

Measure: Administrative costs as a percent of total program costs.

Explanation:This "overhead rate" measure is not a true efficiency measure but is a meaningful surrogate used for all DOE applied R&D and related programs. The objective is to maintain a reasonable overhead rate for effective operation while ensuring that the vast majority of funds address the program purpose. Administrative costs include all Program Direction costs plus costs for supporting activities and analysis funded through programmatic appropriations, including program support. The targets and actuals represent corporate figures (i.e., for all Fossil Energy Research and Development (R&D)) because some Fossil Energy R&D Program Direction costs are difficult to parse at the program level in a meaningful way. Appropriation levels for Fossil Energy R&D programs and for Fossil Energy R&D Program Direction directly affect whether the target is achieved.

Is the program purpose clear?

Explanation: The program goal is to accelerate technology development for exploration and production of nonconventional resources, to advance technology development aimed at maximizing the productivity of each well and to reduce the impact of production. For methane hydrates, the purpose is to understand the role of gas hydrates in seafloor stability and the global carbon cycle, and to develop, by 2015, the knowledge and technology necessary for commercial production of methane from hydrates while protecting the environment. This goal is shared by the five agencies that cooperate on gas hydrate research and development (R&D): The Departments of Energy (DOE), Commerce, Interior, Defense and the National Science Foundation (NSF).

Evidence: Budget documentation (e.g., FY 2004 Budget Congressional Justification); web pages; workshop proceedings; Methane Hydrate Research and Development Act of 2000 PL 106-193; Methane Hydrates Strategic Plan; and Multi-Year R&D Plan.

Does the program address a specific and existing problem, interest, or need?

Explanation: The gas industry has stated that "Sufficient resources exist to meet growing demand well into the twenty-first century." However an ongoing National Petroleum Council Natural Gas Study is revisiting this issue. While the industry dentifies technology development as important, the need for Federal involvement isn't clear, since the industry's average annual capital budget for E & P was over $27 billion from 1991 to 1998 and is expected to grow. The FY 2004 President's Budget request refocused the program on longer-term, higher risk efforts such as hydrogen from natural gas, sustainable supply, and ultra-deep drilling.

Evidence: National Petroleum Council "Natural Gas: Meeting the Challenges of Adding Reserves".

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: The program funds projects comparable to those funded by private industry, and generally for the benefit of private industry. The Methane Hydrate program is designed to make a unique contribution. It addresses research to develop domestic hydrate reserves. It is unique in the US in aiming toward future production. Other Federal agencies address other aspects of gas hydrates(e.g. United States Geological Survey (USGS), resource assessments; National Oceanic and Atmospheric Administration, biota of subsea hydrates.) DOE's long-term reserach on hydrates is beyond the capital investment horizon of most companies.

Evidence: FY 2004 DOE R&D Investment Criteria submission for Oil Exploration and Production (E&P) states: "The independent operator's business model approach (including the largest independents) is to 'buy technology from the service companies' as needed." This statement, which also applies to the gas production business, illustrates that a market for these technologies exists, and that DOE research is often duplicative of, or competes with and potentially crowds out, private investement. See also the Hydrate Strategic Plan and Interagency Coordination Plan.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that an alternative design would be more efficient or effective at developing pre-competitive knowledge and technology. For the Hydrates program, tax incentives for production have been shown to be ineffective in stimulating industry in the absence of needed technology. The international cooperation in the research underscores the importance and approval of the approach.

Evidence: FY 2004 R&D Investment Criteria submission; www.netl.doe.gov/publications/proceedings/00/hydrates/00hydrate.html (Note the original hydrate strategy (see Section 1, Q 1) was developed and critiqued at two national hydrate workshops).

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: DOE has not presented R&D Investment Criteria information at a detailed level discussing variables such as years to commercialization, public benefits, technological risk, cost share or plotting economic, environmental and/or security beneifts.

Evidence:  

Does the program effectively articulate potential public benefits?

Explanation:  

Evidence:  

If an industry-related problem, can the program explain how the market fails to motivate private investment?

Explanation:  

Evidence:  

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Long-term goals are: 1) by 2015, develop technologies to expand the 2002 domestic gas economically recoverable resource base by 100 trillion cubic feet (Tcf); 2) develop technologies that will by 2025 add 20 Tcf of technically recoverable resources of natural gas from methane hydrates; 3) by 2013, reduce the cost of hydrogen production from natural gas by 25% from current baseline of $5.54/MM Btu (steam reforming of methane at Natural gas price of $3.15/MM Btu.)

Evidence: See "Measures" section of this PART; Joule System.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Ambitious targets and timeframes have been established for long-term measures. Projects last for three to five years, with new competition required at that point. New proposals are judged against new proposals.

Evidence: See "Measures" section of this PART.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: Achievement of the annual measures (as well as long term goals) for this program is assessed through the GSAM model. Further detailed analysis is necessary to ensure that the critical assumptions in this model are valid and transparent, and to ensure that any resource base expansion is actually attributable to DOE activities. Additionally, annual measures for methane hydrates need to be agreed upon.

Evidence: See "Measures" section of this PART.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: Program has established baselines and ambitious targest for annual measures.

Evidence:  

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Each cooperative agreement between DOE and participating partners has detailed milestones and key decision points. These milestones are reviewed annually between the DOE management and the performing organization, as well as at all project funding decision points. If a partner is not meeting stated requirements, then that cooperative agreement does not to go into the next budget period. A Federal Advisory Committee and an Interagency Coordinating Committee oversee, coordinate, and integrate Methane Hydrates research to a common mission. Regular national and international meetings of researchers strengthen understanding of the DOE goal. The National Academy of Sciences National Reserach Council (NAS/NRC) is planning to start periodic peer reviews of the program in 2003. All funding awards focus on a statement of work related to program goals.

Evidence: PROMIS project management database; program solicitations; quarterly JOULE Milestones. The Methane Hydrate program's work is tied to the goals and oversight requirements in the Methane Hydrate R&D Act of 2000.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program was recently reviewed by the NAS/NRC. However, the program has previously used industry reviews to evaluate effectiveness and performance. These efforts do not meet the requirements for independent review. The program will explore including third-party reviews. The Methane Hydrate Federal Advisory Committee conducts meetings to address issues that may arise. The NAS/NRC will start periodic peer reviews in 2003.

Evidence: NAS/NRD report: "Energy Research at DOE: Was it Worth It?" (July 2001).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Department has not submitted budget documents linking performance goals to resource levels in a complete or transparent manner.

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Improvements in benefits modeling, and efforts to connect long and short term goals through the JOULE performance tracking system are concrete steps that could help with planning efforts.

Evidence: JOULE System, budget documents.

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: The program did not submit R&D Investment Criteria information demonstrating how DOE prioritizes programs based on potential benefits.

Evidence: Public Workshop Proceedings, EIA NEMS benefit analysis.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: Program did not submit R&D Investment Criteria information demonstrating how risk, years to commercialization, and other key factors are used in setting priorities. However, the GSAM model is used to prioritize gas projects based on supply impact and R&D success.

Evidence: GSAM

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The cooperative agreements in the gas program clearly outline the major milestones and performance requirements that the participating partner must meet. These milestones are tracked and performance measured through quarterly and other technical reporting requirements. Accomplishments and key milestones are reported weekly to the Laboratory Director. Significant accomplishments are transmitted to the Assistant Secretary of Fossil Energy (FE) and are reported in technical fact sheets (TechLines), available to the public on the DOE/FE web site. Project reviews are held on an annual basis. JOULE is used as a performance management tool to track results on a quarterly basis.

Evidence: Progress is tracked through progress, management, and financial reports from the participating partner required by the cooperative agreement. Project databases (ProMis) facilitate and inform project and program management decisions. Websites (such as www.netl.doe.gov/scng) have information regarding the projects and programs. The SCNG website contains information regarding the reporting requirements of cooperative agreements.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: The National Energy Technology Laboratory (NETL) has identified a schedule of incentives holding key product personnel responsible for results under their control. Recently, NETL was one of only two organizations in the entire Federal Government to win Office of Personnel Management's Pillar Award for outstanding efforts in linking performance with accountability. However, the program has not demonstrated that an earned value system for tracking cost, schedule, and performance currently exists.

Evidence: OPM's Pillar Award for linking performance with accountability.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds are obligated based on an annual Program Implementation Plan approved by DOE management and consistent with appropriations. Essentially all funds are obligated in the budget year. Spending of cooperative agreements is audited as required by procurement rules.

Evidence: Program implementation plan signed by the Director of NETL and the Deputy Assistant Secretary includes all planned funding obligations and a schedule of procurements; DOE's annual Performance and Accountability reports.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has not demonstrated that proceedures exist to measure and achieve efficiencies and cost effectiveness in program administration. Although Headquarters staff has been reduced by 13% over the past five years, DOE did not demonstrate cost-savings from this, or highlight whether field/lab staff has shown efficiencies.

Evidence:  

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program is fairly well integrated with other programs with similar goals at DOE, and with non-Federal parties. Methane hydrates R&D is integrated through the interagency coordination committee consisting of representatives from the Departments of Energy, Interior, Commerce, Defense and the National Science Foundation (NSF).

Evidence: NETL Operations Plan, Program plans, including the Methane Hydrates Interagency Coordination Plan www.netl.doe.gov/scng/hydrate/pdf/InteragencyPrint.pdf

Does the program use strong financial management practices?

Explanation: DOE received clean audits in FY 2001 and FY 2002; no known deficiencies specific to this program. Several computer-based project management controls are in place to assist in financial management. Systems exist both on the financial side and the project management side.

Evidence: DOE annual Performance and Accountability report.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Recent and continuing efforts to develop adequate performance measures is a positive step. As the "Top to Bottom" review is fully implemented additional areas will be addressed.

Evidence: Fossil Energy Top-to-Bottom Review, JOULE results.

Does the program allocate funds through a competitive, merit-based process, or, if not, does it justify funding methods and document how quality is maintained?

Explanation: About 97 percent of program funds are subject to competive selection.

Evidence: Selection Process for In-house R&D Projects; proceedings/summary of peer review meetings; annual in-house work proposals

Does competition encourage the participation of new/first-time performers through a fair and open application process?

Explanation:  

Evidence:  

Does the program adequately define appropriate termination points and other decision points?

Explanation:  

Evidence:  

If the program includes technology development or construction or operation of a facility, does the program clearly define deliverables and required capability/performance characteristics and appropriate, credible cost and schedule goals?

Explanation:  

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: Between 1978 and 2000, the gas program has provided over 4 Tcf of incremental production (a subset of economically recoverable resources) or about 1.4 % of consumption.

Evidence: NAS/NRC report: "Energy Research at DOE: Was it Worth it?" (2001)

Does the program (including program partners) achieve its annual performance goals?

Explanation: Achievement of annual measures (as well as long term goals) is based on measurement using the GSAM model. Further detailed analysis is necessary to ensure that the critical assumptions in the model are valid and transparent, and to ensure that resource base expansion is actually attributable to DOE. Under old measures, program has 100% performance score through 2nd quarter of FY 2003.

Evidence: JOULE results, ProMIS project database of accomplishments.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: Program did not demonstrate improved efficiencies or cost effectiveness in achieving program goals.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation:  

Evidence:  

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Incremental production attributed to program by NAS/NRC study. However, the NAS/NRC also stated that it is difficult to accurately attribute DOE contributions versus private industry contributions.

Evidence: NAS/NRD report: "Energy Research at DOE: Was it Worth It?" (2001).

If the program includes construction of a facility, were program goals achieved within budgeted costs and established schedules?

Explanation:  

Evidence: