Detailed Information on the
Vehicle Technologies Assessment

Develop guidance that specifies a consistent framework for analyzing the costs and benefits of research and development investments, and use this information to guide budget decisions.

Provide funds for a peer review of 21st Century Truck Partnership activities. The peer review will include an assessment of the appropriateness of Federal support in each program area.

Consider recommendations from the FreedomCAR peer review and take appropriate budgetary and management action.

Set priorities and identify decision points to focus resources on solving the most critical problems to commercialization of technologies that can reduce petroleum consumption.

Consider recommendations from the FreedomCAR and 21st Century Truck peer reviews and take appropriate budgetary and management action.

Measure: "Parasitic" losses (e.g., aerodynamics, cooling, compressed air) in heavy vehicles. (Reducing parasitic losses can improve fuel economy.)

Explanation:Improving fuel economy contributes to the Department's goal of improving energy security by developing technologies that can improve energy efficiency. The figures are based on modeled projections from individual investigators assuming technical feasibility. The technolgocial advances may or may not be cost effective and commercialized by industry.

Measure: Internal combustion laboratory demonstrated engine efficiency for light-duty vehicles (%).

Explanation:Department's goal of improving energy security by developing technologies that can improve energy efficiency. Engine efficiency improvements can improve vehicle fuel economy. Figures represent engine tests and not necessarily engines that are commercially available.

Measure: Internal combustion engine efficiency for light-duty vehicles. (Engine efficiency improvements can improve vehicle fuel economy.)

Explanation:Improving vehicle engine efficiency contributes to the Department's goal of improving energy security by developing technologies that can improve energy efficiency. Figures represent engine tests and not necessarily engines that are commercially available.

Measure: Internal combustion laboratory demonstrated engine efficiency for heavy-duty vehicles while meeting EPA 2007 and 2010 emissions standards (%)

Explanation:Improving vehicle engine efficiency contributes to the Department's goal of improving energy security by developing technologies that can improve energy efficiency. Engine efficiency improvements can improve vehicle fuel economy.

Measure: Modeled production cost of high-power, 25 kW lithium-ion battery

Explanation:Storage batteries are a key cost and performance component for hybrid vehicles, which offer improved fuel economy. The model and associated results used to demonstrate progress will be validated through external independent review.

Measure: Weight of an unloaded tractor/trailer. (Reducing the weight of a tractor/trailer can increase fuel economy of heavy-duty trucks.)

Explanation:Improving vehicle fuel economy contributes to the Department's goal of improving energy security by developing technologies that can improve energy efficiency. Figures represent modeled weight and not necessarily technology that would be commercially available. Congressional earmarks in 2006 significantly reduced funding for this activity.

Measure: Modeled weight reduction of a passenger vehicle body and chassis system relative to 2002 baseline (%).

Explanation:Improving vehicle fuel economy contributes to the Department's goal of improving energy security by developing technologies that can improve energy efficiency. Reducing the weight of the body and chassis can increase the fuel economy of passenger vehicles. Figures represent modeled weight reduction on a standardized vehicle to allow year-over-year comparisons. The light-weighting technology would not necessarily be commercially available at the time of reporting, but the intent is to reduce vehicle weight safely and cost effectively. The modeling used to demonstrate progress will be validated through external independent review.

Measure: Administrative costs as a percent of total program costs (%)

Explanation:This "overhead rate" measure is not a true efficiency measure but is a meaningful surrogate used for all DOE applied R&D and related programs. The objective is to maintain a reasonable overhead rate for effective operation while ensuring that the vast majority of funds address the program purpose. Administrative costs include all Program Direction and Program Support costs plus costs for supporting activities and analysis funded through programmatic appropriations. The targets and actuals represent corporate figures (i.e., for the entire Office of Energy Efficiency and Renewable Energy) because some EERE Program Direction costs are difficult to parse at the program level in a meaningful way. Appropriation levels for EERE programs and for EERE Program Direction directly affect whether the target is achieved. The baseline and targets for this measure are under development.

Is the program purpose clear?

Explanation: The Vehicle Technologies (VT) Program mission is to develop more energy efficient and environmentally friendly highway transportation technologies that enable America to use less petroleum. The long-term aim is to develop "leap frog" technologies that will provide Americans with greater freedom of mobility and energy security, with lower costs and lower impacts on the environment.

Evidence: EERE Strategic Plan, p. 12 www.eere.energy.gov/office_eere/pdfs/fy02_strategic_plan.pdf

Does the program address a specific and existing problem, interest or need?

Explanation: The Vehicle Technologies program develops hybrid vehicle and other advanced automotive technologies and materials that can improve fuel economy of light- and heavy-duty vehicles in the near term and can contribute to the development of hydrogen fuel cell vehicles in the long-term. Highway vehicles account for 54% of the Nation's oil use (the largest of any sector). The U.S. imports more than half of the oil it consumes, and imports are expected to increase absent major technological improvements or policy changes.

Evidence: Transportation Oil Gap Analysis conducted by EIA and presented in the National Energy Policy (2001).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Nearly all of the program's activities are coordinated with vehicle manufacturers and suppliers through either the FreedomCAR partnership or 21st Century Truck partnership. Program partners collaborate on pre-competitive research beyond the scope of what the industry would do alone. The program coordinates as appropriate with other agencies (DoD, DOT, EPA) conducting similar or related work.

Evidence: FreedomCAR Partnership Plan

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program coordinates with industry to develop vehicle technologies that can improve fuel efficiency and lead to reduced oil consumption. A regulatory program (Corporate Average Fuel Economy, or CAFE) and a tax incentive program for hybrid/fuel cell vehicles (changes/extensions pending) currently exist. However, there is no evidence that either of these approaches alone would be a more efficient means to achieve the goal.

Evidence:  

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: DOE currently estimates benefits of its applied R&D programs, but needs to improve consistency in methodology and assumptions help allocate resources based on costs and benefits. Furthermore, external peer reviews curently underway may help assess whether some program resources may benefit the private sector.

Evidence: FY 2005 EERE GPRA Benefits Estimates p. 13-14 through 3-16 (www.eere.energy.gov/office_eere/gpra_estimates_fy05.html)

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has established three long-term performance measures. These measures are focused on program outputs (not process-oriented) that could enable significant oil savings. However, success is dependant on the rate and level of market penetration, which are strongly affected by other policy instruments (such as fuel economy standards) and market conditions (such as fuel prices). The program also tracks progress against additional goals with its industry partners from the FreedomCAR and 21st Century Truck partnerships.

Evidence: Vehicle Technologies Multi Year Program Plan (draft Nov. 2003), p. 3-1 through 3-8.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The VT Program has articulated ambitious but achievable technical targets associated with the long-term measures.

Evidence: FreedomCAR Partnership Plan, p. 9-12, www.eere.energy.gov/vehiclesandfuels/pdfs/program/freedomcar_partnership_plan.pdf); Vehicle Technologies Multi Year Program Plan (draft Nov. 2003), p. 4-27 through 4-30; EERE FY 2005 Congressional Budget, Vehicle Technologies, p. 258-259

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Long-term performance measures can either be measured directly on an annual basis or are supported by a limited number of annual targets that are discrete, quantifiable, and relate to outputs that can enable reduced oil consumption. Many additional measures and targets not reported in this PART are represented in internal program documents.

Evidence: EERE FY 2005 Congressional Budget, Vehicle Technologies, "Annual Performance Results and Targets," p. 258-259; DOE Summary Program Plans, Vehicle Technoloies, p. 3-8

Does the program have baselines and ambitious targets for its annual measures?

Explanation: All measures have baselines and ambitious targets.

Evidence: EERE FY 2005 Congressional Budget, Vehicle Technologies, Annual Performance Results and Targets, p. 258-259. www.eere.energy.gov/office_eere/budget.html; Office of FreedomCAR and Vehicle Technologies Multi-year Program Plan, p. 4-44

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The FCVT Program's approach to planning and implementing its R&D programs emphasizes jointly funded partnerships with industry to develop and validate technologies. These teams and industry interactions have provided input to the program's R&D roadmaps and portfolio priorities. Awards of grants are predicated on awardee's contractual commitment to overall performance goals and to meeting specific progress targets.

Evidence:  

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: A FreedomCAR review (conducted by the National Academy of Sciences [NAS] through the DOE IG's Office) will be completed in 2005; a similar review is planned for the 21st Century Truck Partnership for 2006. In additional to technical merit, these reviews will address the appropriateness of the Federal support for program activities. DOE highway transportation research had been reviewed annually (1994 -2001) by the NAS and other peer groups. Periodic peer and merit reviews are held for projects at the national laboratories and for contractor projects. EERE policy as of FY 2004 is for programs to have a minimum of biennial reviews at the project level. A peer review of the VT draft multiyear plan was held in December 2003. Project peer reviews are organized by the program, but the panel is comprised of industry and university experts who recuse themselves in cases of potential conflicts of interest.

Evidence: Review of the Research Program of the PNGV, Seventh Report, 2001, p.9-12; Peer Review of the Office of FreedomCAR and Vehicle Technologies Multi-year Program Plan, Dec. 2003; Merit Review and Peer Evaluation of FY 2003 DOE Advanced Combustion Engine R&D, May 2003.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget activities are linked directly to performance goals and targets and program technical needs. Program direction costs, i.e. the cost of implementing and managing the technical effort, is approximately proportional to the sub-program technical budgets although this has not been explicitly stated in the budget.

Evidence: EERE FY 2005 Congressional Budget, Vehicle Technologies.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: EERE programs dynamically evaluate their strategic planning processes. For example, the recent EERE restructuring included a comprehensive evaluation of this program's analytical capabilities and the creation of an EERE-wide planning and analysis team. Similarly, recommendations by peer reviewers have been incorporated into program plans. For example, a key recommendation from the final PNGV peer review led to DOE's redefining of the PNGV program activities and objectives, which led to the FreedomCAR partnership.

Evidence: Presentation slide, EERE FY 2006 Pre-budget Summit, "(OFCVT) Strategic Actions"; EERE Strategic Action Plan (outlines corrective action steps to address deficiencies defined by the 2000 NAPA review). Review of the Research Program of the PNGV, Seventh Report, "Selected Recommendations", recommendation #1 p. 11-12.

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: DOE currently estimates benefits of its applied R&D programs, but needs to improve consistency in methodology and assumptions help allocate resources based on costs and benefits. The program cannot currently make meaningful comparisons of its costs and benefits to other programs with similar goals.

Evidence: EERE GPRA Benefits Estimates, www.eere.energy.gov/office_eere/gpra_estimates_fy05.html

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The program uses a prioritization process for making budget decisions, relying heavily on peer review results. The program also determined that there are new rationale-based decision tools that would serve to enhance decision making and provide clearer documentation. The program is currently evaluating these tools for future implementation.

Evidence: OFCVT Decision Process for Research and Development, February 2004

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The EERE Strategic Management System -- which establishes at the beginning of each fiscal year a 18-month schedule for key planning, budget formulation, budget execution, and analysis / evaluation functions -- requires that each EERE program establish and track long-term and near-term program performance milestones on a monthly and quarterly basis. Program results are used annually and throughout the year to assess partners' performance.

Evidence: DOE Summary Program Plans, Vehicle Technologies, p. 4-8; DOE Joule performance tracking system

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Performance Appraisal and Management Plan for each Technology Development Manager includes an element to provide technical direction to industry, laboratories and universities to support the achievement of program level milestones and to keep projects on schedule and within cost. Solicitations identify the key technology challenges and associated technical targets. Proposals are evaluated on their ability to contribute solutions to these key technology challenges. Projects (including cooperative agreements and national laboratory efforts) are negotiated to include milestones and go/no-go decision points that support the achievement of the program performance goals.

Evidence: Performance Plan and Performance Appraisal Form for Performance Management System Employees; Sample contract documents.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Each year, the program develops an Annual Operating Plan, which is reviewed internally to ensure that new funding is planned to be obligated consistent with the appropriated purpose. EERE also develops a Spend Plan for all of its programs. The program uses data from Departmental procurement and financial systems -- and similar data from National Laboratory partners -- to assure that actual expenditures occur for intended purposes and on a schedule consistent with the Spend Plan. The program reports that cooperative agreement invoices are paid only after verification that the costs are in accordance with the approved budget.

Evidence: FY 2003 Annual Operating Plan. FY 2004 Apportionment. FY 2003 Spend Plan.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: EERE's reorganization in 2002 clarified lines of responsibility and eliminated organizational "stovepipes" by consolidating planning, budgeting, and analysis into a single business administration office. The reorganization reduced management layers, although staff levels remained the same. EERE developed a new IT report to improve program managers' access to EERE cost, obligation, and procurement data. EERE plans to consolidate several legacy IT systems into a single program management system that is intended to track all required information on a project by project basis (cost share, type of contract according to A-11 definitions, etc.). EERE has developed a measure to reduce uncosted balances, i.e. obligated funds will be put to use more quickly. This should achieve efficiency and improve cost effectiveness.

Evidence: EERE Reorganization "All Hands" presentation: www.eere.energy.gov/office_eere/pdfs/eere_reorg.pdf; IT Investment Plan (EERE IT Business Case Number 019-20-01-12-01-1011-00-304-101); www.eere.energy.gov/office_eere/budget.html.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program coordinates closely with the Hydogen, Fuel Cell, and Infrastructure Technology Program to ensure its FreedomCAR activities complements related work supporting the President's Hydrogen Fuel Initiative. Technical activities among EERE Offices having common needs are being coordinated through crosscut teams so as to leverage these efforts and avoid unneeded duplication. The proram also coordinates as appropriate with related programs at DoD, NASA, EPA, and DOT.

Evidence: FreedomCAR Partnership Plan www.eere.energy.gov/vehiclesandfuels/pdfs/program/freedomcar_partnership_plan.pdf; Renewable Fuels Crosscut Team: 21st Century Truck Partnership (members list)

Does the program use strong financial management practices?

Explanation: Each year, EERE develops and maintains a Spend Plan and a Measures spreadsheet that links the Spend Plan to annual and long-term goals and measures for each EERE program. The program requires monthly cost reports that are evaluated against progress and used the make financial adjustments. The program works with the NETL Field Office to manage cooperative agreements, audit partners, ensure invoices are in accordance with the agreements and to issue reimbursement. There is no evidence of erroneous payments or statutory violations.

Evidence: FY 2003 Spend Plan and Measures spreadsheet. Sample quarterly costing report.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The National Association of Public Administrators (NAPA) found dozens of management deficiencies in the program's bureau (EERE) in a review published in 2000. EERE provided evidence that it addressed some of management deficiencies identified by NAPA, and has prepared a Management Action Plan to address many of the remaining findings. While a few NAPA recommendations have not been addressed (e.g., that EERE conduct periodic audits to assure that cost-sharing partners actually provide funding they agree to), in general, EERE has taken meaningful steps to address most deficiencies. At the program level, the program has drafted a Management and Operations Plan that links the research, development, demonstration, and education activities to policies, requirements and the process for selecting options; organizing the program; and managing and monitoring the program. The program also developed a Systems Integration Plan to provide a disciplined approach to the design, development, and validation of complex systems.

Evidence: A Review of the Management in the Office of Energy Efficiency and Renewable Energy (NAPA, 2000). Letter Report from Assistant Secretary Garman to Chairman of the House Subcommittee on Interior and Related Agencies on Implementation of NAPA recommendations (July 11, 2001). EERE Management Action Plan (August 2003). "Reorganizing for Results (Final NAPA on EERE Management), September, 2004.

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: Competitive procurement processes are used for all financial assistance awards, with the exception of earmarks when they occur, such that all proposed activities are evaluated for scientific and technical merit. In addition, the program conducts annual reviews using the EERE scorecard criteria to guide peer evaluations as well as merit reviews by DOE project leaders and staff. These reviews are used to monitor the progress and quality of funded activities. Program funding is tracked and allocated through a funding data base such as the Program Spend Plan.

Evidence: Fiscal Year Spend Plan and Measurements Spread Sheet. Sample solicitation.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: A "large extent" is supported by recent performance against targets and other progress, such as the transfer of technology to commercial use. While the program has met most of its recent performance targets, a full "yes" answer requires a longer track record.

Evidence: EERE FY 2005 Congressional Budget, Vehicle Technologies, Annual Performance Results and Targets p. 257-258, 270, 280 www.eere.energy.gov/office_eere/budget.html; Annual project reviews (e.g. FY 2003 Advanced Combustion Engine R&D Annual Progress Report; Merit Review and Peer Evaluation of FY 2003 DOE Advanced Combustion Engine R&D, December 2003 www.eere.energy.gov/vehiclesandfuels/resources/fcvt_publications.shtml; Review of the Research Program of the PNGV, Seventh Report, 2002, p. 2-10;

Does the program (including program partners) achieve its annual performance goals?

Explanation: Program perfomance on three of annual performance goals are listed on the "Measures" tab. Additional progress is documented in annual progress reports, independent peer reviews, and through other sources. In additional to annual goals, progress also includes the transfer of technology to commercial use.

Evidence: EERE FY 2005 Congressional Budget, Vehicle Technologies, Annual Performance Results and Targets p. 258-259, www.eere.energy.gov/office_eere/budget.html; Annual project reviews; e.g. FY 2003 Advanced Combustion Engine R&D Annual Progress Report, www.eere.energy.gov/vehiclesandfuels/resources/fcvt_publications.shtml; Review of the Research Program of the PNGV, Seventh Report,2002, "Progress and Major Achievements", p. 2-10.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Department has set as its efficiency measure for this program to reduce the ratio of program direction/management funding (DOE and comparable contractor costs) to total program funding. The reduction will be based on a completed assessment of fully defined and costed program management activities currently underway. The baseline and targets are under development.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation:  

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations have shown that the program is generally effective in achieving technical targets. The most recent National Academy of Sciences peer review of the PNGV program (2002) suggested that the program would probably not achieve its 2004 goal of a passenger vehicle capable of 80 mpg, but the report also criticized the goal.

Evidence: Review of the Research Program of the PNGV, Seventh Report,2002, "Progress and Major Achievements", p. 2-10; Merit Review and Peer Evaluation of FY 2003 DOE Advanced Combustion Engine R&D, May 2003.