Detailed Information on the
Hydrogen Technology Assessment

Working to develop a budget presentation that more clearly allocates overhead costs and adequately displays the full cost of the program's activities.

Establishing an adequate baseline and targets for the program's efficiency measure.

Develop guidance that specifies a consistent framework for analyzing the costs and benefits of research and development investments, and use this information to guide budget decisions.

Measure: Modeled cost of hydrogen produced from natural gas (at 5,000 pounds per square inch (psi), untaxed, at the pump, with no carbon sequestration), in dollars per gasoline gallon equivalent ($/gge).

Explanation:Reducing hydrogen production costs accelerate the market viability and deployment of hydrogen technologies, which contribute to the Department's goal of increased energy security and reduced greenhouse gas and pollutant emissions. The outyear targets have slipped from their original levels because Congressional earmarks have reduced funds available to support work that contributes to the goal.

Measure: Modeled cost of hydrogen produced from renewables (at 5,000 pounds per square inch (psi), untaxed, at the pump), in dollars per gasoline gallon equivalent ($/gge).

Explanation:Reducing hydrogen production costs accelerate the market viability and deployment of hydrogen technologies, which contribute to the Department's goal of increased energy security and reduced greenhouse gas and pollutant emissions. The increase of the FY 2005 actual value of modeled cost of hydrogen produced from renewables is due to two factors: (a) increase in the assumed industrial electricity price from 5 cents/kWh from the EIA Annual Energy Outlook (2004 vs. 2005) and (b) increase of capital cost estimate of electrolyzer. Targets and status post 2005 are based on distributed reforming of renewable liquids. Previous targets and status were based on electrolysis, which will not likely be a major renewable technology when used in distributed applications with grid power. In addition, the post-2005 timeline has been extended consistent with reduced funding available for renewable production due to previous years' appropriations and Congressionally-directed projects.

Measure: Hydrogen storage capacity (specific energy) based on capacity (specific energy) based on modeled full-scale system using advanced storage technologies, in weight percent. (Six weight percent will enable a 300-mile driving range in some vehicles.)

Explanation:Reducing hydrogen storage volume will enable the development of hydrogen fuel cell vehicles that are competitive with gasoline powered vehicles for driving range. The metric is for weight, but volume, performance and cost targets must also be met. The 2015 target will enable a driving range of greater than 300 miles across the full range of light-duty vehicles. The 2006 current status was the modeled capacity based on a sub-scale complex metal hydride storage system.

Measure: Average percentage of correct answers by a sample of the general public to technical-knowledge questions regarding hydrogen and fuel cells.

Explanation:Educating the public about hydrogen properties, applications, and safety can help facilitate a more rapid transition to the widespread use of hydrogen and fuel cell technologies.

Measure: Validated cost of hydrogen production, untaxed, in dollars per gasoline gallon equivalent ($/gge).

Explanation:This measure tracks demonstration activities intended to validate modeled or estimated costs of hydrogen production. Reducing hydrogen production costs can help accelerate the market viability and deployment of hydrogen technologies, which contribute to the Department's goal of increased energy security and reduced greenhouse gas and pollutant emissions. "Validated" means demonstrated at operational scale for a given period of time. The 2004 and 2005 targets include co-production of electricity at 8 cents/kWh. The 2009 and 2013 targets do not include co-production of electricity.

Measure: Annual displacement of petroleum, in millions of barrels of oil per day (MM b/d)

Explanation:As technical targets are met in this and other related programs, hydrogen fuel cell vehicles and other hydrogen-powered technologies may become commercially viable, thereby displacing oil consumption. Values may be adjusted annually as more information about competing technologies and alternative vehicles are included in the analysis. The original assessment of 11 MMBPD was based on hydrogen fuel cells as the primary option to replace gasoline vehicles in the mass market. The updated value of 2.1 MMBPD reflects oil savings due to hydrogen while also accommodating competing technologies such as hybrid and biofuels vehicles.

Measure: Percent of codes and standards developed to enable initial market introduction of hydrogen-powered fuel cell vehicles and infrastructure.

Explanation:The development and promulgation of codes and standards are essential to establish a market-receptive environment for commercial hydrogen and fuel cell products and systems. This measure tracks the results of the Department's safety R&D as well as coordination of and support to various organizations that prepare critical codes and standards needed to expedite hydrogen infrastructure development and the emergence of hydrogen as a significant energy carrier.

Measure: Modeled technology cost of a hydrogen-fueled 80kW automotive fuel cell power system ($/kW)

Explanation:Reducing automotive fuel cell costs accelerate the market viability and deployment of fuel cell technologies, which contribute to the Department's goal of increased energy security and reduced greenhouse gas and pollutant emissions.

Measure: Administrative costs as a percent of total program costs (%).

Explanation:This "overhead rate" measure is not a true efficiency measure but is a meaningful surrogate used for all DOE applied R&D and related programs. The objective is to maintain a reasonable overhead rate for effective operation while ensuring that the vast majority of funds address the program purpose. Administrative costs include all Program Direction and Program Support costs plus costs for supporting activities and analysis funded through programmatic appropriations. The targets and actuals represent corporate figures (i.e., for the entire Office of Energy Efficiency and Renewable Energy) because some EERE Program Direction costs are difficult to parse at the program level in a meaningful way. Appropriation levels for EERE programs and for EERE Program Direction directly affect whether the target is achieved. The baseline and targets for this measure are under development.

Is the program purpose clear?

Explanation: The mission of the DOE's Hydrogen Program is to research, develop, and validate fuel cell and hydrogen production, delivery, and storage technologies. The long-term aim is to accelerate progress toward an energy future for the Nation where hydrogen plays a more significant role as an energy carrier in all sectors of the economy and all regions of the country, so that environmental and energy security benefits can be realized.

Evidence: Hydrogen Posture Plan (December 2006). FY 2004-2008 Budgets. Energy Policy Act of 2005. Hydrogen Futures Act of 1996.

Does the program address a specific and existing problem, interest, or need?

Explanation: The hydrogen program develops hydrogen-based technologies for transportation and electricity production in an effort to reverse America's growing dependence on foreign oil, enhance energy diversity and energy security, and reduce greenhouse gas emissions. These potential benefits support some of the goals included in the Energy Policy Act of 2005 (EPACT 2005), as well as the Administration's climate change goals. The program is a key component of the President's Hydrogen Fuel Initiative announced in January, 2003 and Advanced Energy Initiative announced in January, 2006.

Evidence: The program focuses R&D on activities that it considers too technologically risky for the private sector to undertake alone. Risk levels vary on a project-by-project and technology-by-technology basis. The purpose of the Hydrogen Program as reauthorized in EPACT 2005 is included in Section 802. Information on the Advanced Energy Initiative is available on line at: www.whitehouse.gov/stateoftheunion/2006/energy/energy_booklet.pdf

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program collaborates with other Federal agencies, State agencies, industry groups, and non-profit organizations to avoid duplication of efforts. The program participates in monthly meetings of the Interagency Hydrogen and Fuel Cell technical Task Force (established in December 2005 in response to EPACT 2005) to better coordinate hydrogen-related activities among relevant Federal agencies. At the request of the Secretary in 2003 a Hydrogen Posture Plan was collaboratively developed with the Department's Offices of Energy Efficiency & Renewable Energy (EERE), Fossil Energy (FE), Nuclear Energy (NE), and the Office of Science (SC) and with the Department of Transportation (DOT). The Plan helps coordinate planned hydrogen-related activities within the Department. A Hydrogen Fuel Initiative Coordination Group was formed in 2004 to continue the coordination within the Department on Hydrogen related issues. The coordination group includes representatives from EERE, FE, NE, SC, and DOT. The National Hydrogen Energy Roadmap, released on November 12, 2002, was developed by approximately 220 technical experts and industry practitioners from public and private organizations.

Evidence: National Hydrogen Energy Roadmap (November 2002). Hydrogen Posture Plan (December, 2006). DOE Hydrogen Program Management and Operations Plan (September 2005). Hydrogen Fuel Initiative Coordination Group. The Hydrogen program participates in monthly Hydrogen Coordinating Meetings, which bring together program managers and research leads for the Department's EERE, FE, SC, and NE offices. The program aims to accelerate private sector efforts on hydrogen technologies for which markets do not yet exist.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that another approach or mechanism would be more efficient or effective to achieve the intended purpose. The long term goal of using hydrogen as a fuel for cars and trucks to help reduce dependence on oil cannot be achieved without advances in hydrogen and fuel cell research. Regulations can play a facilitating role by providing codes and standards specifications for hydrogen and fuel cell technologies. EPACT 2005 (section 782, 783) also authorizes federal procurement incentives for vehicles as well as stationary and portable fuel cells but reliable hydrogen and fuel cell technologies must be available before such options are widely implemented. Such regulations cannot mandate the development of an entirely new energy source. The same is true for market-incentive policies, which require the technology basis for targeted markets to develop. At this early stage in the R&D cycle for hydrogen and fuel cell technologies, alternative policy options cannot substitute for or adequately induce the needed technology improvements.

Evidence: The program found no studies that indicate a tax credit, regulatory driver, or other policy mechanism would be a more cost effective approach than R&D, since most of the technologies being researched are so far from commercialization. Scenario analysis activities have indicated that policy instruments will be important in the commercialization phase of hydrogen technology. National Academies Peer Review. Hydrogen Posture Plan (December 2006). Hydrogen Technical Advisory Committee (HTAC) reviews. EPAct (Title VIII and section 782, 783). Hydrogen Program Multi-Year Research, Development and Demonstration Plan (2007)

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program currently estimates the potential benefits of its applied R&D programs but needs to improve consistency in methodology and assumptions across and within programs to help target resources based on costs and potential benefits. Progress has been made in improving benefits estimations and methodologies and the program continues to work with other DOE programs to further improve consistency and accuracy. The program's benefit estimates are part of a larger EERE and DOE benefit estimation process. The program engages in cost shared research and demonstration activities with the universities, auto and energy companies, state and local governments, and hydrogen and fuel cell technology developers. It also supports research at national labs.

Evidence: EERE benefit estimates, available on line at http://www1.eere.energy.gov/ba/pba/gpra.html. In general, the program appears to target its resources appropriately within the program, but a lack of appropriate evidence mandates a No response. The program is contributing to a Department-level effort to address the issue.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has developed one key outcome measure (displacing petroleum). The program can track progress against this measure directly, but significant progress will not occur until major technical barriers that the program's R&D addresses are overcome. The program also has several output measures that cover most of the programs key activities, including R&D on the production of hydrogen from renewable and non-renewable resources, hydrogen storage, automotive fuel cell cost, infrastructure validation, codes and standards, and public education efforts.

Evidence: FY 2008 Budget; Hydrogen Program Multi-Year Research, Development, and Demonstration Plan (February, 2007); Hydrogen Posture Plan (December 2006)

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Targets and timeframes are quantitative and ambitious. The program's multi-year research identifies R&D technical targets, baselines, off-ramps, and endpoints. Projects (including cooperative agreements and national laboratory efforts) are negotiated to include milestones and go/no-go decision points that support the achievement of the program performance goals.

Evidence: Hydrogen Program Multi-Year Research, Development, and Demonstration Plan (February, 2007). Hydrogen Posture Plan (December, 2006). Annual operating plan and cooperative agreement example demonstrating inclusion of go/no-go decision points.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Most of the long-term measures can be tracked directly on an annual basis, and therefore can also be considered annual measures. The program also has milestones for its projects, each of which directly support a long-term goal. Output measures are used to track key activities that support the achievement of program goals, including R&D on the production of hydrogen from renewable and non-renewable resources, hydrogen storage, automotive fuel cell cost, infrastructure validation, codes and standards, and public education efforts.

Evidence: FY 2004 - FY 2008 Budgets, Hydrogen Program Multi-Year Research, Development, and Demonstration Plan (February, 2007). Hydrogen Posture Plan (December 2006).

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Baselines have been established for all annual performance measures. Targets appear to be ambitious and based on technology gaps, industry needs, stakeholder input and program goals. A National Academies' review also considered the program's planned achievements challenging.

Evidence: Multi-Year Research, Development, and Demonstration Plan (2007). Hydrogen Posture Plan (December, 2006). National Academies Peer Review: Review of the Research Program of the FreedomCAR and Fuel Partnership: First Report, (Washington, DC: National Academies Press, 2005, page 4). FY2004 - FY2008 Budgets.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The majority of contractors are funded based on commitment to the subprogram's annual performance goals through a competitive, cost-shared solicitation process. Long-term goals are developed in partnership with a wide spectrum of public and private industry representation. Industry partners in the Administration's FreedomCAR partnership have committed to the partnership's goals. Project contributions to annual/long-term goals are provided at Annual Merit and Peer Evaluation meetings and in Annual Progress Reports.

Evidence: FreedomCAR Partnership Plan (April, 2003). Sample competitive solicitation. Sample Cooperative Agreement Statement of Objectives and National Laboratory Annual Operating Plan and Work Plan. Annual Merit Review and Peer Evaluation (2006 example). Annual Progress Report (2006 example).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program regularly conducts external merit reviews of its R&D projects. An annual report is published that includes recommendations for each project. In addition, the National Research Council (NRC) is currently reviewing the FreedomCAR and Fuel Partnership and the program's strategies and progress. The NRC has completed 2 previous independent reviews (of the program and of the FreedomCAR and Fuel Partnership), which offered a number of recommendations, such as addressing: a systems approach to hydrogen energy RD&D; exploratory research as the foundation for breakthroughs in technology; safety issues; and coordination of R&D strategy and programs. These reviews were of sufficient scope to cover the entire program. The program has a track record of being responsive to the recommendations of program evaluators and continues to work to address recommended actions as they are identified. The program also conducts independent technical reviews such as the 2006 review verifying the achievement of $3/gge for hydrogen production from distributed natural gas.

Evidence: National Research Council and National Academy of Engineering, Committee on Alternatives and Strategies for Future Hydrogen Production and Use, The Hydrogen Economy: Opportunities, Costs, Barriers and R&D Needs, (Washington, DC: National Academies Press, 2004)_;_National Research Council of the National Academies; Committee on Review of the FreedomCAR and Fuel Research Program, Phase 1; Board on Energy and Environmental Systems, Division on Engineering and Physical Sciences, Transportation Research Board; Review of the Research Program of the FreedomCAR and Fuel Partnership: First Report, (Washington, DC: National Academies Press, 2005). DOE Hydrogen Program Annual Merit Review and Peer Evaluation (May 16-19, 2006; May 23-26, 2005; May 24 - 27, 2004). Annual Merit Review and Peer Evaluation Report (e.g. 2006 as example). NRC Letter Report (April, 2003). Independent Review of Hydrogen Production from Distributed Natural Gas (October, 2006; http://www.hydrogen.energy.gov/pdfs/40382.pdf). Independent Review of Fuel Cell Cost (October 2006).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget documents do not clearly indicate the full costs of achieving program goals and it is not always clear from the documents how changes in funding, policy, or legislative decisions could impact expected performance on program goals. Administrative expenses are presented as a separate line item for an entire office in which this program resides; it is not clear how the administrative funding is allocated to the various programs it supports. In general, budget documents do identify the resources needed to achieve its goals and do include a table listing key activities that support meeting program goals.

Evidence: FY 2008 Budget

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program developed and updated a detailed Multi-year Research, Development, and Demonstration Plan. The program led the development of the Hydrogen Posture Plan, in collaboration with the Offices of Fossil and Nuclear Energy, with input from the Office of Science and the Department of Transportation, to ensure planning is aligned with Departmental strategic priorities. This high level strategic plan was updated and published in December 2006. The Department established the Hydrogen Technical Advisory Committee (HTAC) which reviewed the program's strategic plans. In addition, at the program's request, the National Academies conducted independent reviews of the program. Finding from these reviews informed planning processes. The program has responded to the Academies' recommendations.

Evidence: Hydrogen Program Multi-Year Research, Development, and Demonstration Plan (February, 2007). Hydrogen Posture Plan (December, 2006). Hydrogen Program Annual Merit Review and Peer Evaluation Report (2006 example). National Academies' evaluation reports and response to findings documents.

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: The program has not yet developed a consistent, reliable methodology for comparing potential benefits within and across programs with similar goals. The program estimates some benefits of its activities in support of the Government Performance and Results Act and the Administration's Research and Development Investment Criteria initiative. The program's efforts to estimate benefits are part of a larger office/department-wide effort to implement an estimation process that uses common assumptions and baselines. The Department has made progress in its efforts, but further improvement is still required. Once a consistent, reliable methodology has been demonstrated, the results must be validated by an independent entity.

Evidence: EERE benefit estimates, available on line at http://www1.eere.energy.gov/ba/pba/gpra.html. The program is contributing to a Department-level effort to address shortfalls in developing a consistent methodology for benefits estimation.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The program defines technical priorities based both on market needs and detailed trade-off analyses between system requirements, performance and cost. Projects are judged annually by a combination of process analysis, and independent peer review to assess needs and performance against objectives. In addition, for example, workshops with industry experts and Nobel laureate scientists were conducted to support the development of targeted solicitations on hydrogen storage. The program participated in an EERE-wide Integrated Priorities List budget exercise in which priorities at the activity level were clearly laid out.

Evidence: Hydrogen Program Multi-Year Research, Development, and Demonstration Plan (February 2007). EERE Priority Ranking Tool; Integrated Priorities List.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Corporate Planning System (CPS) requires that the Hydrogen program (along with all other EERE programs) establish and track long-term and near-term performance goals and measures. Program results as evaluated through the goals and measures are used annually and throughout the year to assess partners' performance, adjust funding, and re-align R&D portfolios. The program also conducts annual peer review meetings and regular meetings with the industry partners (e.g., FreedomCAR & Fuel Partnership) technical teams to review projects and assess performance towards meeting the program goals. The program also conducts independent analysis of the various technological solutions. Using this process, the program has made a "no go" decision on on-board fuel processing because analysis indicated this technology would not be competitive and did not meet performance requirements.

Evidence: EERE Corporate Planning System. Report on go/no-go decision for on-board fuel processing (provided as example). The program also reports on quarterly milestones in the Department's internal Joule database. However, in general, milestones in the Joule system are not necessarily fully reflective of overall program progress as assessed by PART measures. The Department's is working to improve PART-Joule measure alignment. The program also tracks current status and progress compared to technical targets in its Multi-Year Research, Development and Demonstration Plan (2007) target tables, which is an effective method for publicly communicating progress towards targets in the R&D community. The new I-MANAGE system, currently under development, is supposed to better integrate budget and performance. Annual Merit Review Meetings.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Performance Appraisal and Management Plan for each Technology Development Manager includes an element to provide technical direction to industry, laboratories and universities to support the achievement of program level milestones and to keep projects on schedule and within cost. Solicitations identify the key technology challenges and associated technical targets. Proposals are evaluated by technical experts on their ability to contribute solutions to these key technology challenges. Projects (including cooperative agreements and national laboratory efforts) are negotiated to include milestones and go/no-go decision points that support the achievement of the program performance goals.

Evidence: Performance Plan and Performance Appraisal Form for Performance Management System Employees; Sample contract documents.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Each year, the program develops a Spend Plan through the Department's Corporate Planning System (CPS), which is reviewed internally to ensure that new funding is planned to be obligated consistent with the appropriated purpose and in a timely manner. The intended purpose of the funds along with relevant milestones and Annual Operating Plans for National Laboratories are all described in the CPS database. The program uses data from Departmental procurement and financial systems -- and similar data from National Laboratory partners -- to assure that actual expenditures occur for intended purposes and on a schedule consistent with the Spend Plan. The program reports that cooperative agreement invoices are paid only after verification that the costs are in accordance with the approved budget. Adequate procedures exist for reporting actual expenditures and the program's unobligated balances are low.

Evidence: Hydrogen Technology Program Monthly Financial Status reports (April 2007). FY 2007 Apportionment. FY 2007 program Spend Plan. The program's unobligated balances from FY 2005 and FY 2006 that were carried forward to FY 2007 were $607,313.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not have an adequate baseline for its efficiency measure. All Energy Efficiency and Renewable Energy (EERE) programs, including Hydrogen, have established an overhead rate efficiency measure however the proposed baseline for that measure does not capture funding associated with the full breadth of program management activities. Once a baseline and targets have been established, the measure will be useful because consistently tracking the overhead rate and ensuring that it's within a reasonable range should help to improve efficiency. To help improve efficiency, the program's bureau (EERE) has developed a suite of new reports and tools which all program uses to improve managers' access to cost, obligation, and procurement data. EERE is consolidating several legacy IT systems into a single program management system designed to track all required information on a project by project basis (cost share, type of contract according to A-11 definitions, etc.).

Evidence: EERE Corporate Planning System (CPS). The program is contributing to a Department-level effort to establish a consistent baseline for its efficiency measure.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program coordinates informally with EERE's wind, solar, biomass, and federal energy management programs; and formally through Memoranda of Understanding (MOU) with NASA, the Department of Agriculture, and the Department's Office of Fossil Energy (FE). The program coordinates continuously with the Office of Science (SC), such as through workshops, technical conferences and contractor meetings. The program coordinates a formal monthly coordination group involving the Offices of EERE, SC, FE, NE and DOT. The program also participates in a Hydrogen R&D Interagency Task Force to better coordinate hydrogen-related activities among relevant Federal agencies. The program led the formulation of a Hydrogen Posture Plan (updated and published December 2006), which was collaboratively developed with the Department's Offices of Energy Efficiency & Renewable Energy, Fossil Energy, and Nuclear Energy, with input from the Office of Science and from the Department of Transportation. The Plan helps coordinate planned hydrogen-related activities within the Department. The program also coordinates and collaborates with relevant international programs, such as through the International Partnership for the Hydrogen Economy(IPHE) and International Energy Agency (IEA) to leverage global expertise, avoid duplication and minimize U.S. funding requirement.

Evidence: Hydrogen Posture Plan (December, 2006). MOUs with Office of Fossil Energy, USDA and NASA. Hydrogen R&D Interagency Task Force Schedule of Meetings and Agendas (2006). Example joint workshops through the International Partnership for a Hydrogen Economy (www.iphe.net).

Does the program use strong financial management practices?

Explanation: Each year, EERE develops and maintains a Spend Plan and a Measures spreadsheet (part of the Congressional budget request which corresponds to the "Joule" measures). Projects in the Spend Plan are linked to annual and long-term goals and measures for each EERE program. The program compiles monthly cost reports that are evaluated against progress and used the make financial adjustments. The program works with the Golden Field Office to manage cooperative agreements, audit partners, ensure invoices are in accordance with the agreements and to issue reimbursement. There is no evidence of erroneous payments or statutory violations.

Evidence: FY 2006 and FY 2007 Spend Plans. Monthly cost reports (e.g. April 2007 and Sept 2006). FY 2006 Performance and Accountability Report.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: In a review published in 2000, the National Association of Public Administrators (NAPA) found dozens of management deficiencies in the program's bureau (the Office of Energy Efficiency and Renewable Energy, or EERE). EERE provided evidence that it addressed some of management deficiencies identified by NAPA, and has prepared a Management Action Plan that will address many of the remaining findings. While a few NAPA recommendations have not been addressed (e.g., that EERE conduct periodic audits to assure that cost-sharing partners actually provide funding they agree to), in general, EERE has taken meaningful steps to address most deficiencies. At the program level, the program has drafted a Management and Operations Plan that links the research, development, demonstration, and education activities to policies, requirements and the process for selecting options; organizing the program; and managing and monitoring the program. The program also developed a Systems Integration Plan to provide a disciplined approach to the design, development, and validation of complex systems. In addition, at the program's request, the National Academies has conducted two reviews of the program, which outlined a number of recommendations that the program has responded to.

Evidence: A Review of the Management in the Office of Energy Efficiency and Renewable Energy (NAPA, 2000). Letter Report from Assistant Secretary Garman to Chairman of the House Subcommittee on Interior and Related Agencies on implementation of NAPA recommendations (July 11, 2001). EERE Management Action Plan (August 2003). DOE Hydrogen Program Management and Operations Plan (September 2005). Systems Integration Plan (2003). The Hydrogen Economy: Opportunities, Costs, Barriers and R&D Needs, (Washington, DC: National Academies Press, 2004); National Research Council of the National Academies; Committee on Review of the FreedomCAR and Fuel Research Program, Phase 1; Board on Energy and Environmental Systems, Division on Engineering and Physical Sciences, Transportation Research Board; Review of the Research Program of the FreedomCAR and Fuel Partnership: First Report, (Washington, DC: National Academies Press, 2005). Program responses to the findings and recommendations in these reports.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Historically, earmarks have prevented all program grants from being awarded through a clear competitive process. In 2006, approximately 28% percent of the program's funding ($43 million out of $153 million total) was earmarked. However, as a result of the 2007 appropriations process, this temporarily improved and virtually all program funds were awarded competitively. For non-earmarked grant funds the Hydrogen Program uses open, competitive solicitations as the primary mechanism for its cost-share in effective public-private partnerships to accomplish specific RD&D targets. The solicitation process is streamlined and uniform across all Hydrogen Program activities. Proposal reviews follow the DOE Merit Review Guide.

Evidence: FY 2007 appropriations act Public Law 110-5 (H.J. Res 20) and DOE's FY 2007 Spend Plan. DOE Financial Assistance Regulation 10 CFR Part 600. DOE Merit Review Guide for Financial Assistance and Unsolicited Proposals. EERE Merit Review Procedures. Sample solicitation documentation showing competitive process for labs.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program follows DOE's Guide to Financial Assistance (March 2005). Each grantee is required to provide regular technical and financial status reports. For most grantees, the status reports are due quarterly. EERE's Project Management Center (PMC) reviews and documents its assessment of progress and financial performance. The PMC forwards the status reports to the HQ program office for their review on a quarterly basis. The program and PMC project officers regularly conduct site visits, teleconferences and review meetings with the grantees. The program conducts an annual merit review at which progress is presented and projects undergo peer review. In addition, the program compiles an annual progress report for every project.

Evidence: DOE's Guide to Financial Assistance, March 2005. Sample technical and financial staus reports. Proceedings for the Annual Merit Review.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program collects, compiles, and disseminates grantee performance information in an accessible manner. The program's website provides the proceedings of its annual merit review. The proceedings provide information on a project-by-project basis. Presentations, complete annual progress reports, and reviewer comments and scores are all available on the website.

Evidence: Annual merit review proceedings available at http://www1.eere.energy.gov/hydrogenandfuelcells/annual_review.html

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: Competitive Procurement Processes are used for all financial assistance awards (except for most earmarks), such that all proposed activities are evaluated for scientific and technical merit during the selection process. The program conducts an annual peer review using criteria to guide peer evaluations. Projects are scored and either continued, discontinued or revised in work scope, based on the annual peer review. In FY 2006, approximately 28 percent of the funds were earmarked. The program competitively awarded the majority of the remaining R&D funds, primarily using the established merit review/external (peer) evaluation processes.

Evidence: FY 2006 and FY 2007 Spend Plan. Sample solicitation documents demonstrating competitive procurement processes. Table showing funding allocations as per OMB Circular A-11 definitions for "Conduct of Research and Development." Hydrogen Program Annual Merit and Peer Evaluation Report (Table showing project scores), 2004, 2005, 2006.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Generally, the program appears to be making reasonable progress, although some measures and targets are new. In addition, the National Academies' reviews noted that the program has made significant R&D advances. For example, two independent reviews assessed the program's performance in reducing the cost of hydrogen production from distributed natural gas and in reducing the cost of fuel cells.

Evidence: National Academy of Sciences, "Renewable Power Pathways: A Review of The U.S. Department of Energy's Renewable Energy Programs" (2000). The Hydrogen Economy: Opportunities, Costs, Barriers and R&D Needs, (Washington, DC: National Academies Press, 2004)_;_National Research Council of the National Academies; Committee on Review of the FreedomCAR and Fuel Research Program, Phase 1; Board on Energy and Environmental Systems, Division on Engineering and Physical Sciences, Transportation Research Board; Review of the Research Program of the FreedomCAR and Fuel Partnership: First Report, (Washington, DC: National Academies Press, 2005). Independent Review of Hydrogen Production from Distributed Natural Gas (October, 2006; http://www.hydrogen.energy.gov/pdfs/40382.pdf); Fuel Cell System for Transportation- 2005 Cost Estimate (October, 2006; http://www.hydrogen.energy.gov/pdfs/40160.pdf)

Does the program (including program partners) achieve its annual performance goals?

Explanation: Generally, the program appears to be making reasonable progress, although some measures and targets are new. The program reports that it is on track toward achievement of its FY 2007 targets. In addition to PART, the program reports annual achievements in DOE's Performance and Accountability Report.

Evidence: FY 2008 Budget (see pp. 65-68 fir Joule measures status). PART measures available at http://www.whitehouse.gov/omb/expectmore/. Performance Accountability Report FY 2006.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has adopted an administrative overhead measure to assess efficiency. However, an adequate baseline has not been established. The program has undertaken other efforts, such as IT improvements, to improve efficiency.

Evidence: The program received a No in question 3.4 as a result of not having an adequate baseline for the efficiency measure and therefore a No in this question is also required. The program is contributing to a Department-level effort to address the issue.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The program works closely with industry and other Federal programs to advance the state of the art in hydrogen technologies. There are no studies comparing this program to similar programs.

Evidence: n/a

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: According to a National Academy of Sciences review, the program is "well defined and well managed...The research itself appears to be well done, and an organized peer review system is in place." The program "has established a firm foothold in critical technical areas that can provide incremental improvements." Other peer reviews generally report positively on technical progress of projects.

Evidence: National Academy of Sciences, "Renewable Power Pathways: A Review of The U.S. Department of Energy's Renewable Energy Programs" (2000). : National Research Council and National Academy of Engineering, Committee on Alternatives and Strategies for Future Hydrogen Production and Use, The Hydrogen Economy: Opportunities, Costs, Barriers and R&D Needs, (Washington, DC: National Academies Press, 2004)_;_National Research Council of the National Academies; Committee on Review of the FreedomCAR and Fuel Research Program, Phase 1; Board on Energy and Environmental Systems, Division on Engineering and Physical Sciences, Transportation Research Board; Review of the Research Program of the FreedomCAR and Fuel Partnership: First Report, (Washington, DC: National Academies Press, 2005). DOE Hydrogen Program 2006 Annual Merit Review and Peer Evaluation (May 16-19, 2006). DOE Hydrogen Program 2005 Annual Merit Review and Peer Evaluation (May 23-26, 2005). DOE Hydrogen Program 2004 Annual Merit Review and Peer Evaluation (May 24 - 27, 2004). 2003 Merit Review and Peer Evaluation of the Hydrogen, Fuels Cells and Infrastructure Technologies Program (May 19-22, 2003). 2002 Annual Hydrogen Program Review Meeting (May 6-8, 2002). Merit Review and Peer Evaluation National Laboratory R&D (May 9-10, 2002).