Detailed Information on the
Industrial Technologies Program Assessment

Develop guidance that specifies a consistent framework for analyzing the costs and benefits of research and development investments, and use this information to guide budget decisions.

Contract with an independent organization to review the program's assessment of its contribution to the long-term goal of improving industrial energy efficiency.

Measure: Percent reduction in energy intensity (energy used per value of product shipments) for eight energy intensive industries.

Explanation:This single long-term measure captures the purpose of the program and directly links to the Department's strategic goal to improve energy security by improving energy efficiency. The measure assumes all energy intensity improvements are attributable to the program, although the program recognizes that the Energy Information Administration estimates that more than half of the improvements would occur under a "business-as-usual" scenario without the program. Other factors may also affect achievement of targets. For example, if the market value of the products increase, the energy intensity could decrease, without a real change in the energy used per product.

Measure: Number of technologies commercialized, in partnership with the most energy-intensive industries, that improve energy efficiency of an industrial process or product by at least 10 percent.

Explanation:The program partners with industry to develop energy efficient products and processes, and industry is required to cost share in the research. Therefore, it is expected that new, more efficient products will penetrate the market and improve industrial energy efficiency, contributing to the Department's strategic goal to improve energy security by improving energy efficiency. When a technology's full-scale unit is operating in a commercial setting and it is available for sale or license, it is considered commercialized.

Measure: Administrative costs as a percent of total program costs (%).

Explanation:This "overhead rate" measure is not a true efficiency measure but is a meaningful surrogate used for all DOE applied R&D and related programs. The objective is to maintain a reasonable overhead rate for effective operation while ensuring that the vast majority of funds address the program purpose. Administrative costs include all Program Direction and Program Support costs plus costs for supporting activities and analysis funded through programmatic appropriations. The targets and actuals represent corporate figures (i.e., for the entire Office of Energy Efficiency and Renewable Energy) because some EERE Program Direction costs are difficult to parse at the program level in a meaningful way. Appropriation levels for EERE programs and for EERE Program Direction directly affect whether the target is achieved. The baseline and targets for this measure are under development.

Measure: Estimated energy savings from technical assistance activities such as industrial plant-wide assessments, facility audits, and training activities, in trillions of British thermal units (TBtus).

Explanation:One trillion Btus is equivalent to the energy contained in about 166,000 barrels of oil. The measure assesses performance of the program's deployment activities. The measure extrapolates energy savings from each technical assistance activity based on historic data. This approach allows assessment of program performance within the year, as opposed to measuring actual energy saved from implementation of recommendations, some of which may have been made in earlier years. Recommended projects or process improvements have different lag times for implementation. Industry's implementation of recommended energy efficiency improvements will contribute to the Department's strategic goal to improve energy security by improving energy efficiency.

Is the program purpose clear?

Explanation: The mission of the Industrial Technologies Program is to decrease the energy intensity of the U.S. industrial sector through a coordinated program of research and development, validation, and dissemination to provide industry with energy-efficient technologies and operating practices. The program carries out this mission through partnering with industry, its equipment manufacturers, and its many stakeholders.

Evidence: FY 2006 OMB Budget Request; program publications. Authorizing legislation includes: P.L. 94-163, "Energy Policy and Conservation Act" (EPCA) (1975); P.L. 94-385, "Energy Conservation and Production Act" (ECPA) (1976); P.L. 95-91, "Department of Energy Organization Act" (1977); P.L. 95-619, "National Energy Conservation Policy Act" (NECPA) (1978); P.L. 95-620, "Powerplants and Industrial Fuel Use Act of 1978"; P.L. 96-294, "Energy Security Act" (1980); P.L. 101-218, "Renewable Energy and Energy Efficiency Technology Competitiveness Act of 1989"; P.L. 102-486, "Energy Policy Act of 1992".

Does the program address a specific and existing problem, interest, or need?

Explanation: The President's National Energy Policy (NEP) seeks to improve the overall energy intensity of the U.S. economy, recommending that the President "direct the Secretary of Energy to establish a national priority for improving energy efficiency. The priority would be to improve the energy intensity of the U.S. economy as measured by the amount of energy required for each dollar of economic productivity. This increased efficiency should be pursued through the combined efforts of industry, consumers, and federal, state, and local governments." Research, development, and outreach that the program conducts on industrial energy efficiency helps address this NEP recommendation. The NEP recognizes that reducing energy intensity in American industries can result in improvements in industrial productivity, product quality, safety, and pollution prevention. Industry is the Nation's largest energy-consuming sector, accounting for about one-third of all U.S. energy use.

Evidence: The President's National Energy Policy (NEP), 2001

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program has developed an effective strategy to engage specific industries in collaborative partnerships to address pre-competitive research needs, involving the joint development of national "visions of the future," and the "technology roadmaps" for getting there. Industry identifies its full suite of technology needs for the future, within which the Federal government's activities can be clearly delineated. EERE works closely with States and research institutions, such as universities, to introduce large energy-consuming industrial plants to energy savings technologies developed through ITP partnerships, as well as ITP-developed software tools and related training and other practices. The National Research Council (NRC) applauded DOE's approach in two reports, but cautioned that the program's research agenda should not become too applied and focused on the short-term, owing the natural tendency of industry to do so.

Evidence: NRC, Decreasing Energy Intensity in Manufacturing: Assessing the Strengths and Future Directions of the Industrial Technologies Program, 2005. NRC, Energy Research at DOE: Was It Worth It?, 2001.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The National Research Council noted that "(t)he strategy of partnering with the end user of the technology to determine overall program goals, set grand challenges, and share in the project costs is a proven success." (p. 80) There is no evidence that other approaches, such as regulation or tax incentives, would be more effective or efficient means of reducing the energy intensity of U.S. industries.

Evidence: NRC, Decreasing Energy Intensity in Manufacturing: Assessing the Strengths and Future Directions of the Industrial Technologies Program, 2005.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: DOE currently estimates the potential benefits of its applied R&D programs, but needs to improve consistency in methodology and assumptions across and within programs to help target resources based on costs and potential benefits. External peer reviews have not assessed whether some program resources may benefit the private sector without producing public benefits or in the absence of a market failure (i.e., lack of incentive for industry investment). The program is working to address these issues. In addition, the program focuses on energy-intensive industries and calculates the magnitude of the gap between current energy use in major processes and the theoretical minimum energy requirements to establish technology focus areas addressing the largest opportunities in each industry.

Evidence: ITP's Strategic Plan, studies of the gaps between current energy use and the theoretical and practical minima chemicals, steel, and metal casting industries,

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program's long-term performance goal is: By 2020, contribute to a 20 percent reduction in energy intensity (energy used per value of product shipments compared to 2002) in the energy-intensive Industries of the Future. This goal relates directly to the program's purpose: to improve the energy efficiency of the U.S. industrial sector. It is important to note that the measure assumes all energy intensity improvements are attributable to the program, although the program recognizes that the Energy Information Administration estimates that more than half of the improvements would occur under a "business-as-usual" scenario without the program. Other factors may also affect achievement of targets. For example, if the market value of the products increase, the energy intensity could decrease, without a real change in the energy used per product.

Evidence: FY 2006 Congressional Budget

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The targets and timeframes are ambitious yet reasonable and achievable. The targets for improvement in energy intensity through 2020 follow a fairly straight path. The program must continue to help commercialize energy efficient products and recommend industrial process improvements that industry implements in order to achieve the 2020 goal.

Evidence: FY 2006 Budget includes the energy intensity goal statement.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has two annual measures: 1. The number of technologies that the program helps to commercialize that improve energy efficiency of an industrial product or process by at least 10 percent; 2. The estimated energy savings from the program's technical assistance activities. These annual output measures tie to the long-term goals and support the program purpose of improving energy efficiency of the U.S. industrial sector. The second measure is calculated formulaically based on the number of program activities (people trained, facility audits conducted, etc.), so there is some uncertainty in the results. It should also be noted that progress in achieving the long-term outcome measure is reported on annually, so this measure also serves as an annual measure.

Evidence: These annual measures are modified versions of measures that the Department has identified in its 2006 Budget.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The baselines and targets for the annual measures are reasonable. The target for the number of technologies that the program plans to help commercialize that can improve energy efficiency of indutrial processes reflects the state of development promising technologies that the program has been supporting. The target for expected energy saved from technical assistance activities is extrapolated based on historical performance on the suite of activities (plant-wide assessments, facility audits, and training.)

Evidence: FY 2006 Budget

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Sample program contract documents indicate that project performers commit to working toward program goals. In addition, executives of companies and trade groups representing industries targeted by the program have signed Memoranda of Understanding (MOU's) with the Department of Energy. As a result of these MOUs, industries have participated in the development of industry "visions" and "technology roadmaps" in support of the cooperative partnership between industry and the Federal government. These roadmaps are used to set industry-wide R&D priorities and help the program and industry jointly select, implement and review projects contributing to the goals identified in the roadmaps. State and university partners are committed through grants and cooperative agreements to specific milestones that contribute to program goals.

Evidence: Sample contract documents. MOU's with eight energy-intensive Industries of The Future. Industry "Vision" documents and Technology Roadmaps available on line at www.eere.doe.gov/industry

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The National Research Council (NRC) has reviewed the program or one of its activities several times since 1999. Special critical reviews have been undertaken to help redirect subprograms such as the Industrial Assessment Centers and the Industrial Materials of the Future subprogram. Moving forward, the program must follow the Office of Energy Efficiency and Renewable Energy (EERE) Peer Review Review Guide (August 2004), which outlines procedures for independent review. Programmatic reviews are supposed to be conducted at least once every two years. The scope of future reviews should include whether Federal support was warranted for specific projects or activities, and whether there's an appropriate termination point for Federal investment.

Evidence: EERE, Peer Review Guide, 2004. NRC, Industrial Technology Assessments: An Evaluation of the Research Program of the Office of Industrial Technologies, 1999. NRC, Materials Technologies for the Process Industries of the Future: Management Strategies and Research Opportunities, 2000. NRC, Energy Research at DOE, Was It Worth It?, 2001 NRC, Decreasing Energy Intensity in Manufacturing: Assessing the Strategies and Future Directions of the Industrial Technologies Program, 2005.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The budget includes a table that shows funding by Program Goals, of which there is one for the Industrial Technologies Program. The Department's Program Goal is not quantitative and differs from the long-term measures included in this assessment. Nevertheless, it accurately reflects the program mission. In addition, the program is able to estimate administrative costs associated with program implementation, and provides this information in support of its use of "administrative overhead rate" as an efficiency measure.

Evidence: FY 2006 Congressional Budget

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The National Research Council (NRC) found that the program's strategic plan provides a coherent link to the National Energy Policy, the Department's Strategic Plan, the Office of Energy Efficiency and Renwable Energy (EERE) Strategic Plan, and the program's own Multi-Year Program Plan (MYPP), which focuses on "grand challenges" ??important technical challenges facing industry that, if solved, hold the potential to produce large improvements in energy efficiency. The NRC also found that the program has a reasonable five-part "corporate strategy" to help realize the goals. However, the NRC recommended that the program better define how the goals identified in the program's strategic plan were selected and how their achievement will be measured. The goals in the program's strategic plan are similar to long-term goals presented in this assessment. The program will address this recommendation in order to show progress and prove performance in future assessments.

Evidence: NRC, Decreasing Energy Intensity in Manufacturing: Assessing the Strategies and Future Directions of the Industrial Technologies Program, 2005. Industrial Technology Program, Multi-Year Program Plan, 2005.

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: DOE currently estimates benefits of its applied R&D programs, but needs to improve consistency in the methodology and assumptions it uses the help inform resource allocation decisions. The program cannot currently make meaningful comparisons of its costs and benefits to other programs with similar goals.

Evidence: EERE benefits estimates, available on line at www.eere.energy.gov/office_eere/ba/gpra.html

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: ITP's government investment strategy includes prioritization of projects through opportunity analyses and expert peer reviews, maintenance of a robust project portfolio, and synergy with other programs in EERE. The National Research Council (NRC) found that "these tools and processes are appropriate for guiding the prioritization and selection of projects." The program's priorities, in order, are as follows: 1. Industrial Technical Assistance, Steel R&D, and Chemical R&D; 2. Forest and Paper Product R&D, Industrial Materials R&D, Sensors and Automation R&D, Combustion R&D, and Aluminum R&D; 3. Glass R&D, Mining R&D; 4. Metal Casting R&D; 5. Supporting Industry R&D.

Evidence: NRC, Decreasing Energy Intensity in Manufacturing: Assessing the Strategies and Future Directions of the Industrial Technologies Program, 2005. Industrial Technology Program Priority Ranking, July 15, 2005.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: DOE requires that each EERE program establish and track (on a quarterly basis) progress toward long-term and near-term program performance goals. The goals that the program has been reporting on are process-oriented and differ from goals established in this assessment. Nevertheless, the program has provided regular reporting on progress against established milestones through its Joule performance tracking system. The program also collects regularly performance data from project performers. In additions, reported performance is examined across technology areas to identify best practices in program management to improve all technology area performance.

Evidence: Sample quarterly report from DOE Joule performance tracking system. Sample progress report from a contractor: Steelmaking Nozzles that Resist Clogging - Project 0150, April 28, 2005

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Annual Performance Appraisals of all EERE Program Managers include criteria directly related to cost, schedule, and performance results. EERE reviews these criteria monthly in the EERE Monthly Management Reviews. Most EERE contracts include award fee and other performance criteria to hold those partners accountable.

Evidence: Program Managers' Performance Standards; EERE Award Fee and Performance Based contracts

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The FY 2004 and FY 2005 apportionments show that the program had a carryover balances of $4.6 million and $1.1 million, respectively, indicating that more than 95 percent of funds were obligated within each fiscal year. Apportionments from previous years show similar results. However, the program has had high uncosted balances (funding obligated but not yet spent), which is the reason for the "No" response. In FY 2005, program's uncosted balances totaled $29.7 million, representing 40 percent of FY 2005 appropriated funds. While uncosted balances are still high, the balances have decreased significantly in recent years and continue to improve. In addition, each year, the program develops an Annual Operating Plan, which is reviewed internally to ensure that new funding is planned to be obligated consistent with the appropriated purpose. EERE also develops a Spend Plan for all of its programs. The program uses data from Departmental procurement and financial systems and similar data from National Laboratory partners to assure that actual expenditures occur for purposes and on a schedule consistent with the Spend Plan.

Evidence: FY 2004 and FY 2005 apportionments. FY 2005 report on uncosted balances, FY 2004 Industrial Technologies Program Annual Operating Plan, FY 2004 Industrial Technologies Program Spend Plan, sample monthly procurement report.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: All EERE programs, including ITP, have established the EERE corporate overhead rate (i.e., administrative costs to implement all EERE programs) as the efficiency measure. Consistently tracking the overhead rate and ensuring that it's within a reasonable range should help to improve efficiency. In addition, EERE's reorganization in 2002 clarified lines of responsibility and eliminated organizational "stovepipes" by consolidating planning, budgeting, and analysis into a single business administration office. The reorganization reduced management layers, although staff levels remained the same. EERE developed a suite of new reports and tools to improve program managers' access to EERE cost, obligation, and procurement data. EERE is consolidating several legacy IT systems into a single program management system that will track all required information on a project by project basis (cost share, type of contract according to A-11 definitions, etc.).

Evidence: EERE Corporate Planning System (CPS). New "overhead rate" efficiency measure (see also Question 4.3). EERE Reorganization "All Hands" presentation: www.eere.energy.gov/office_eere/pdfs/eere_reorg.pdf. IT Investment Plan (EERE IT Business Case Number 019-20-01-12-01-1011-00-304-101).

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program participates in the Government Agencies Technology Exchange in Manufacturing (GATE-M) with five other federal agencies with manufacturing component in their mission (NIST, DOD, NASA, NSF, NNSA). GATE-M helps agencies exchange technical information and coordinate, collaborate, and leverage resources for manufacturing R&D activities. The Environmental Protection Agency and National Science Foundation have also consulted with the program on how its efforts are conducted. Other governments, such as Canada and the United Kingdom, have met with ITP officials to discuss the structure and implementation of the Industries of the Future subprogram. The program participates in International Energy Agency subgroups (i.e., Working Party on Energy End Use Technology and Working Party on Energy Efficiency).

Evidence: Government Agencies Technology Exchange in Manufacturing (GATE-M) Report., National Institute of Standards and Technology, NISTIR 6950, 2003. Sample implementing agreement and report from participation in International Energy Agency activities.

Does the program use strong financial management practices?

Explanation: Many traditional aspects of accounting and financial management occur within the Departmental CFO Headquarters and Field organizations -- outside of the day-to-day control of EERE's programs. DOE had received clean audit opinions with no material weaknesses for several consecutive years until FY 2005, when transition to a new accounting system disclosed some financial weaknesses, which are being addressed corporately. In response to the review of EERE business management by the National Academy for Public Administration (NAPA), EERE initiated a wide range of improvements, including the development and routine maintenance of the EERE Spend Plan; a Measures spreadsheet that links the Spend Plan to near- and long-term goals and measures; periodic EERE-wide reviews of those processes by the EERE Office of Business Administration, and quarterly costing reports. ITP has pioneered several innovative financial tracking tools, such as a project costing projection worksheet and a project mortgage projection worksheet, both of which are being adopted across EERE. However, DOE's Inspector General found that, for one ITP project, "EERE project officials did not ensure that accounting and business weaknesses disclosed through audits by the Defense Contract Audit Agency were corrected," resulting in questionable expenditures of $359,843, 70 percent of which were reimbursed. On the whole, more than half of the 20 cooperative agreements reviewed "did not receive sufficient management attention." The IG also noted that "EERE officials acknowledged weaknesses in project management and initiated action to improve their project management approach."

Evidence: NAPA, A Review of Management in the Office of Energy Efficiency and Renewable Energy, 2000. ITP's project costing projection worksheet, ITP's project mortgage projection worksheet, ITP Spend Plan, ITP Measures Spreadsheet, sample quarterly costing report.DOE IG Report No. 0689, 2005.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The National Association of Public Administrators (NAPA) found dozens of management deficiencies in the program's bureau (EERE) in a review published in 2000. EERE provided evidence that it addressed some of management deficiencies identified by NAPA, and has prepared a Management Action Plan to address many of the remaining findings. While a few NAPA recommendations have not been addressed (e.g., that EERE conduct periodic audits to assure that cost-sharing partners actually provide funding they agree to), in general, EERE has taken meaningful steps to address most deficiencies. . In 2005, DOE's Inspector General found that, for one ITP project, "EERE project officials did not ensure that accounting and business weaknesses disclosed through audits by the Defense Contract Audit Agency were corrected," resulting in questionable expenditures of $359,843, 70 percent of which were reimbursed. On the whole, more than half of the 20 cooperative agreements reviewed "did not receive sufficient management attention." However, the IG also noted that "EERE officials acknowledged weaknesses in project management and initiated action to improve their project management approach."

Evidence: NAPA, A Review of the Management in the Office of Energy Efficiency and Renewable Energy, 2000. Letter Report from Assistant Secretary Garman to Chairman of the House Subcommittee on Interior and Related Agencies on implementation of NAPA recommendations (July 11, 2001). EERE Management Action Plan (August 2003). DOE IG Report No. 0689, 2005

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The ITP activities use open, competitive solicitations as the primary mechanism as its cost-share in effective public-private partnerships to accomplish specific RD&D targets. The solicitation process is streamlined and uniform across all Industrial Technologies Program activities. A key program strategy is to develop competitive RD&D solicitations in support of the industry-driven roadmaps, and require a minimum 50 percent cost-share from industry over the life of each project. (The program selects projects that address top industry needs, meet strict criteria for government support, are most likely to be replicated, and help meet national security, energy, and environmental policy goals.) Proposal reviews follow the DOE Merit Review Guide.

Evidence: DOE Financial Assistance Regulation 10 CFR Part 600. DOE Merit Review Guide for Financial Assistance and Unsolicited Proposals. EERE Merit Review Procedures, 21 Dec 2001.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The DOE/EERE Merit Review Process is required for awarding all discretionary Financial Assistance Awards. New projects are selected by ITP competitively based on their merits, in particular their potential energy savings impact. After award, ITP uses reviews by both internal and external teams to evaluate efficacy, progress, and merit of funded projects. This is in addition to the oversight provided by the responsible DOE project managers.

Evidence: DOE Financial Assistance Regulation 10 CFR Part 600. DOE Merit Review Guide for Financial Assistance and Unsolicited Proposals. EERE Merit Review Procedures, 21 Dec 2001, PMC documentation.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Information on commercialized (and emerging technologies) have been tracked for ITP by Pacific Northwest National Laboratory for over 20 years. A database of information is maintained and updated annually and is available through a link to the ITP website. In addition, ITP produces annual reports on each technology subprogram which are also available on the ITP website.

Evidence: Impacts report: http://www.pnl.gov/impacts/pdfs/05impacts/front.pdf annual reports for technology subprograms: http://www.eere.energy.gov/industry/news/news_detail.html/news_id=8906

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: The program allocates a portion of its funding each year to National Laboratories. The program issues "Lab Calls" in which National Laboratories compete for available funding. (National Labs by law cannot compete with the private sector and universities, so the Lab Call is separate from public solicitiations.) The program allocates funding for National Laboratoroes because they house world-class facilities where more than 30,000 scientists and engineers perform cutting-edge research. Examples of unique lab capabilities include Brookhaven's high-performance, QCDOC (quantum chromodynamics on a chip) supercomputer, Fermilab's Tevatron collider, and the Berkeley lab's Advanced Light Source user facility, National Energy Research Scientific Computing Center flagship scientific computing facility, and the National Center for Electron Microscopy resources for electron beam microcharacterization of materials.

Evidence: Laboratory Call for Materials for Energy Efficient Industrial Processing, DE-PS36-05GO95024, July 19, 2005.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The key long-term goal is to contribute to a 20 percent decrease in energy intensity by 2020 (as compared to 2002) by the energy intensive Industries of the Future. Both the Industries of the Future subprogram and the Crosscutting Industrial Technologies subprogram can contribute to this goal. The data show that the program is on track to achieve the long-term goal. It should be noted that the program has history of estimating energy savings from its activities, but these estimates have not been externally peer reviewed. The "Large Extent" (as opposed to a "Yes") reflects that there is some uncertainty in the data.

Evidence: Impacts, Industrial Technologies Program: Summary of Program Results for CY 2003 (January 2005). Available on line at: www.pnl.gov/impacts/pdfs/05impacts/front.pdf. This report, prepared by Pacific Northwest National Lab, is an analysis of program activities and results to date.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program does not have a long history of actual performance against targets for its annual measures because the annual measures have been modified from what the Department has used in recent years. Specifically, the measure on the number of technologies commercialized has a new baseline (2004) and now includes a threshold on the improvement in energy efficiency that the commercialized product or process must provide (10 percent). The measure on energy saved from technical assistance activities is new; targets for 2003 through 2006 were back-calculated. Despite the lack of actual performance data against documentable past targets, the program provided data to support that it commercialized energy efficient technologies and saved energy through its technical assistance activities; thus, a rating of Small Extent is appropriate.

Evidence: Impacts, Industrial Technologies Program: Summary of Program Results for CY 2003 (January 2005). Available on line at: www.pnl.gov/impacts/pdfs/05impacts/front.pdf. This report, prepared by Pacific Northwest National Lab, is an analysis of program activities and results to date. DOE Joule reports.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: All EERE programs, including the Industrial Technologies Program, have established the EERE corporate overhead rate (i.e., the ratio of administrative costs to total program costs forimplementing all EERE programs) as the efficiency measure. A Yes response requires that the program demonstrate improved efficiency or cost effectiveness over the prior year. EERE established a baseline in 2004, and set a corporate target for FY 2006 and beyond. However, there is no history of actual performance and therefore no demonstration of whether efficiency over the prior year has improved or remained within an acceptable range. The "Small Extent" rating reflects the fact that EERE has examined closely the use of national laboratories to conduct administrative functions, and has begun to transition some of this work to the private sector and EERE's Project Management Center. EERE projects savings of $4 million in 2005 alone, although there is no independent verification of improved efficiency.

Evidence: EERE, "Transition of Non-R&D National Laboratory Funding to the Private Sector and PMC, Final Report and Interim Plan," April 2005.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There is no other Federal program designed specifically to help industry improve energy efficiency. The program partners with industry, and thus there is no comparable private sector to achieve the same goals in the absence of the Federal program. Some States have programs that may augment the Federal effort, but it's not possible determine the relative impacts of State and Federal efforts on the outcome measure of reducing industrial energy intensity.

Evidence: There is no available documentation comparing the Federal program to State or private sector efforts.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A recent National Research Council (NRC) study found that ITP was "on track to achieve its goals and has an opportunity to exceed them if it continues to focus on improving performance." At the same time, however, the report requested additional information on how the goals were selected and how their achievement will be measured. The report also found ITP to be "a well-managed and effective program" that has a project selection approach that is "solidly-based." In a 2001 NRC review of energy efficiency programs, the NRC found that two of the five ITP-supported technologies reviewed had provided positive economic returns.

Evidence: NRC, Decreasing Energy Intensity in Manufacturing: Assessing the Strategies and Future Directions of the Industrial Technologies Program, 2005. NRC, Energy Research at DOE: Was It Worth It, 2001.