Detailed Information on the
Solar Energy Assessment

Develop guidance that specifies a consistent framework for analyzing the costs and benefits of research and development investments, and use this information to guide budget decisions.

Provide funding consistent with meeting performance targets, and eliminate funding for low-priority earmarks.

Resume limited funding for CSP research and carefully monitor technological progress.

Continue to address technical and management recommendations made by the National Research Council peer review.

Restructure the Applied Research portion of the Photovoltaic subprogram in order to focus on specific PV technology roadmaps (e.g., Thin Film Silicon, Wafer Silicon, CIGS, CdTe, CPV) rather than on general areas of research (e.g., Electronic Materials & Devices, Measurements & Characterization)

Focus Concentrating Solar Power (CSP) R&D on thermal storage technology, and identify other high priority research areas to make the best use of appropriated funds.

Measure: Producer manufacturing cost of photovoltaic modules ($/W)

Explanation:Targets for producer cost of PV modules tie directly to the long-term targets for reducing cost of photovoltaic power. Producer cost data are collected from industry partners. Target unchanged from 2004 to 2005 because program changed from using modeled costs to validated costs based on data provided by industry.

Measure: Cost of energy from solar water heaters in non-freezing climates, in cents per kilowatt-hour (??/kWh).

Explanation:Reducing the cost of solar water heating can result in increased deployment, providing benefits such as reduced emissions from power generation, increased energy supply diversity, reduced energy imports, and increased electricity reiability by reducing the system load on the grid. (Published targets in 1998 and 2000 based on request level; appropriations below request contributed to missed targets.). Prototypes not yet available....

Measure: Modeled levelized cost of power from large-scale concentrating solar power (CSP) plants, in cents per kilowatt-hour (??/kWh). (2008 Actual reflects increase in prices for steel, nitrate salts, heat exchangers and tanks, and pipes/valves & fittings.) Outyear targets based on expected R&D improvements in the receiver and solar collector along with the assumption that commodity prices will moderate to that of the general inflation rate.)

Explanation:The cost targets apply to trough and dish engine technologies, two of the three different CSP technologies.

Measure: Years of durability of polymer materials for solar water heaters, measured by "accelerated" testing.

Explanation:Polymer materials for solar water heaters need to be durable for at least 20 years, in addition to meeting other technical and cost specifications.

Measure: Cost of power from photovoltaics under ideal conditions, in cents per kilowatt-hour (??/kWh). (The cost of energy from a photovoltaic system is dependent on application and system requirements, financing terms, material costs, and other factors, which is why targets are given as ranges. Ideal conditions include sunny locations with easy installation.)

Explanation:The ranges tie to the annual targets for producer cost of PV modules. Reducing the cost of PV modules can result in increased deployment, providing benefits such as reduced emissions from power generation, increased energy supply diversity, reduced energy imports, and increased electricity reliability by reducing the system load on the grid.

Measure: Administrative costs as a percent of total program costs (%).

Explanation:This "overhead rate" measure is not a true efficiency measure but is a meaningful surrogate used for all DOE applied R&D and related programs. The objective is to maintain a reasonable overhead rate for effective operation while ensuring that the vast majority of funds address the program purpose. Administrative costs include all Program Direction and Program Support costs plus costs for supporting activities and analysis funded through programmatic appropriations. The targets and actuals represent corporate figures (i.e., for the entire Office of Energy Efficiency and Renewable Energy) because some EERE Program Direction costs are difficult to parse at the program level in a meaningful way. Appropriation levels for EERE programs and for EERE Program Direction directly affect whether the target is achieved. The baseline and targets for this measure are under development.

Measure: Modeled levelized cost of power for residential photovoltaic markets under ideal conditions (cents/kWh).

Explanation:

Measure: Modeled levelized cost of power for commercial photovoltaics markets under ideal conditions (cents/kWh).

Explanation:

Is the program purpose clear?

Explanation: The program's mission is to develop efficient, reliable, and affordable solar technologies that can transform domestic solar resources into a substantial source of usable energy.

Evidence: EERE FY 2005 OMB Budget Submission. Program first authorized in 1975 by P.L. 94-163, "Energy Policy and Conservation Act" (EPCA). Reauthorized in 1976 (P.L. 94-385), 1978 (P.L. 95-619), and 1992 (P.L. 102-1018).

Does the program address a specific and existing problem, interest, or need?

Explanation: The program aims to expand the use of solar energy, which can increase domestic energy supplies and avoid emissions of pollutants and greenhouse gases associated with conventional methods of power production. These potential benefits support the Administration's National Energy Policy, as well as the Administration's climate change goals.

Evidence: The program focuses R&D on activities that it considers too technologically risky for the private sector to undertake alone. Risk levels vary on a project-by project basis.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: The Solar Energy Program collaborates with industry, academic and State solar research programs, as well as other programs in EERE. From this collaboration, program managers direct research that complements, but does not duplicate, other ongoing efforts. For example, meetings are held yearly with the Energy Materials Coordinating Committee (EMaCC) to review Federal R&D programs conducting similar research within the government. Meetings are also held with State representatives and other organizations, such as the Electric Power Research Institute. Occasionally, other Federal programs, such as the Department of Commerce Advanced Technology Program, fund solar energy projects. However, these efforts are comparatively small and not part of a coordinated research effort.

Evidence: The program considers lack of industry capital to be a market barrier to private sector investment in solar energy technology R&D.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program focuses on reducing costs of solar power though technology development in order to achieve the outcomes of increased domestic energy supply and reduced emissions of pollutants and greenhouse gases.

Evidence: The program found no studies that indicate a production tax credit, regulatory driver, or other policy mechanism would be a more cost effective approach.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The program focuses resources on technologies that are not yet commercially competitive. In support of the Administration's R&D Investment Criteria initiative, the program was asked to prepare "bubble charts" that plot key program variables (e.g., expected public benefits, funding levels, years to commercialization). Bubble charts can serve as an informational tool to help determine, along with other considerations, whether the program appropriately targets its R&D funding. While the program has made progress estimating public benefits, the Department has not yet developed a methodology to estimate benefits consistently within and across programs. Therefore, the program could not prepare meaningful bubble charts.

Evidence: Although unable to prepare bubble charts, the program did estimate years to commercialization for its major R&D activities as follows: photovoltaics (17 years); solar thermal technologies (7 years). The program's estimates have not been peer reviewed. In general, the program appears to target its resources wisely, but a lack of ability to provide appropriate evidence mandates a "no" response. EERE continues to work internally and with other DOE program offices to improve consistency and accuracy in estimating benefits.

Does the program effectively articulate potential public benefits?

Explanation:  

Evidence:  

If an industry-related problem, can the program explain how the market fails to motivate private investment?

Explanation:  

Evidence:  

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program's key long-term measures track cost-of-energy and reasonably represent the most important program activities.

Evidence: FY 2004 Budget. Solar Energy Technology Program. DRAFT Multi-Year Technical Plan (2003).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program's long-term measures are ambitious and designed to maintain aggressive progress. For example, the cost-of-energy goal for photovolatics is 6 cents/kWh by 2020, a significant reduction from the corresponding cost in 2000 of 25 cents/kWh. This is based on new research concepts, e.g., nanostructures and multi-junction cells, that are high risk but could potentially lead to cost breakthroughs. To maintain focus on long-term targets, intermediate goals in periodic solicitations are used to direct and redirect activities within the PV subprogram as well as to redirect the PV subprogram itself. Such decision points provide regular 'on and off ramp' opportunities.

Evidence: Solar Energy Technology Program DRAFT Multi-Year Technical Plan (2003). The US Photovoltaic Industry Roadmap (2001). Photovoltaics, Energy for the New Millennium: The National Photovoltaics Program Plan 2000-2004 (2000).

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: The program's annual performance measures tie directly to the long-term cost-of-energy goals. For photovoltaics, the key annual measure is cost of production of photovoltaic modules. For solar water heating, cost-of-energy is tracked directly. The photovoltaic subprogram should develop additional annual measures that capture its fundamental research activities.

Evidence: FY 2004 Budget. Solar Energy Technology Program DRAFT Multi-Year Technical Plan (2003).

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: The annual performance measures have baselines and are specific and quantified. They are also ambitious but realistic. They are based on estimates from lab engineers and researchers of what can be accomplished in a short period of time, one year, with the available resources.

Evidence: FY 2004 Budget.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The program selects only projects from partners who show commitment and that will contribute to the program goals of decreasing the cost and increasing the performance of systems being developed. Projects are subject to semi-annual or annual reviews, and project performers must submit monthly or quarterly status reports. The majority of performance measures are quantifiable and trends can be linked to objective baselines for the technical performance of the technologies being developed and demonstrated. Program goals are identified when R&D efforts are solicited. Responses from interested parties are evaluated based on the demonstration of their ability to achieve progress toward these goals as presented in work statements submitted for competitive evaluation. Such information is developed as targets and milestones in the final contracting documents.

Evidence: Sample program R&D solicitations.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Each major subprogram is subject to external peer reviews every two years. [The most recent peer review of the concentrating solar power was not independent (one panel member was a member of a solar industry advocacy organization), but that activity was terminated in the FY 2004 Budget.] In addition to peer reviews the program undergoes a thorough internal program review every 18 months. The most recent review was March 2003. The program should consider expanding the scope of peer reviews to include overall program effectiveness and relevance.

Evidence: National Academy of Sciences (NAS), "Renewable Power Pathways: A Review of The U.S. Department of Energy's Renewable Energy Programs" (2000). 2001 Peer Review of the DOE Photovoltaic Program, September 14, 2001. Concentrating Solar Power Peer Review: Final Report (2001). 2001 Peer Review of the U.S. Department of Energy's Solar Buildings Technology Research Program, December 2001.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The budget identifies the resources needed to achieve the program's cost-of-energy performance goals. However, budget documents do not clearly indicate the full costs of achieving the program goals. Salaries, benefits, and other admininstrative expenses to support the program are included in a separate budgetary line item ("Program Direction"). EERE does not report the allocation of Program Direction funding to the various programs it supports.

Evidence: FY 2004 Budget.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: There are no recent reports criticizing the program's strategic planning efforts. The program has consulted with industry and other stakeholders on priority needs and has formulated a multiyear research plan. The program also prepares R&D roadmaps in consultation with industry. One recent PV subprogram responses to PV community needs are the development of the High Performance R&D activity in FY 2000 in response to industry's need for higher efficiency concepts to reduce system costs on a 'per energy' basis. Another example is the development, initiated in 2002, of a concerted effort to address system reliability with particular emphasis on thin-film modules and their special aspects that both promise inexpensive manufacturing processes but also require new approaches to ensure durability.

Evidence: Solar Energy Technology Program DRAFT Multi-Year Technical Plan (2003). The US Photovoltaic Industry Roadmap (2001). Photovoltaics, Energy for the New Millennium: The National Photovoltaics Program Plan 2000-2004 (2000).

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: Each year, the program estimates the public benefits of its activities in support of the Government Performance and Results Act (GPRA) and the Administration's R&D Investment Criteria initiative. However, the program has not yet developed a consistent and reliable methodology for comparing potential benefits within and across programs with similar goals.

Evidence: FY 2004 Congressional Budget Justification materials.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The Solar Energy Program works closely with industry, academic, and State solar research programs to identify R&D needs and prepare "roadmaps" that delineate the highest priority activities that provide the most value. In addition, the program's multi-year program plan defines the major activities that will be carried out over a five-year period. Each activity is assigned a relative impact on system cost, risk of achieving success, and cost. These factors are considered in developing priorities and assigning budgets. In addition, the assessments described in 2.RD1, for example, technological risk, and other factors, such as market potential, are used in establishing the zero-based budget for the Solar Energy Program that identifies priorities at the activities level.

Evidence: Solar Energy Technology Program DRAFT Multi-Year Technical Plan (2003). The US Photovoltaic Industry Roadmap (2001). Photovoltaics, Energy for the New Millennium: The National Photovoltaics Program Plan 2000-2004 (2000). Concentrating Solar Power: An Industry Vision for the New Millennium (2001); Parabolic-Trough Technology Roadmap: A Pathway for Sustained Commercial Development and Deployment of Parabolic-Trough Technology (1999). Concentrating Solar Power Dish Roadmap (2000). Draft: Central Receiver Technology Roadmap: A Pathway for Sustained Commercial Development and Deployment of Central Receiver Technology (2001).

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The EERE Strategic Management System -- which establishes at the beginning of each fiscal year an 18-month schedule for key planning, budget formulation, budget execution, and analysis / evaluation functions -- requires that each EERE program establish and track long-term and near-term program performance goals and measures. Program results as evaluated through the goals and measures are used annually and throughout the year to assess partners' performance, adjust funding, and re-align R&D portfolios.

Evidence: SMS Implementation Letter for FY 2002 - 2005 (October 2001). Monthly, quarterly and annual reports from key program partners and contractors. Performance information on one measure (cost of production of PV modules) is recorded in Joule, the Department's performance management system. However, in general, milestones in the Joule system are not fully reflective of program progress. Thus, the Department's Joule system provides little value-added. The new I-MANAGE system, currently under development, will better integrate budget and performance.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: The Annual Performance Appraisals of all EERE Program Managers include criteria directly related to cost, schedule, and performance results. EERE reviews these criteria monthly in the EERE Monthly Management Reviews. Most EERE contracts include award fee and other performance criteria to hold those partners accountable.

Evidence: Performance Plan and Performance Appraisal Form for Performance Management System Employees; EERE Award Fee and Performance Based contracts.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Each year, the program develops an Annual Operating Plan, which is reviewed internally to ensure that new funding is planned to be obligated consistent with the appropriated purpose. EERE also develops a Spend Plan for all of its programs. The program uses data from Departmental procurement and financial systems -- and similar data from National Laboratory partners -- to assure that actual expenditures occur for intended purposes and on a schedule consistent with the Spend Plan. The program has had year-end amounts ranging from 12 to 35 percent of appropriated funds from FY 2000 to FY 2002. The program reports that the high uncosted level of 35 percent in FY 2002 was due to the anticipation of delayed appropriations for FY 2003. (The program operated did not receive appropriations till halfway through the fiscal year in 2003.) Unobligated balances brought forward to FY 2004 were $337,000, less than one percent of the program's FY 2003 appropriation of approximately $83 million.

Evidence: FY 2003 Annual Operating Plan. Solar Energy Program FY 2003 Financial Status Report (June 2003). FY 2003 Apportionment. FY 2003 Spend Plan.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: EERE's reorganization in 2002 clarified lines of responsibility and eliminated organizational "stovepipes" by consolidating planning, budgeting, and analysis into a single business administration office. The reorganization reduced management layers, although staff levels remained the same. EERE developed a new IT report to improve program managers' access to EERE cost, obligation, and procurement data. EERE plans to consolidate several legacy IT systems into a single program management system that is intended to track all required information on a project by project basis (cost share, type of contract according to A-11 definitions, etc.). EERE is also developing a measure to reduce uncosted balances, which means obligated funds will be put to use more quickly. The program also reports that it has established its own database that track the following information for each project: objectives, background, approach, recipient, location, milestones, status, funding level, and program priority. The database will reportedly be compatible with Departmental databases under development. These recent actions should achieve efficiencies and improve cost effectiveness, although it will be difficult in some cases to demonstrate definitively.

Evidence: EERE Reorganization "All Hands" presentation: www.eere.energy.gov/office_eere/pdfs/eere_reorg.pdf. EERE IT Business Case Number 019-20-01-12-01-1011-00-304-101. Solar Energy Program FY 2003 Financial Status Report (June 2003). Solar Energy Technologies Program database.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program collaborates with related EERE programs, specifically Buildings Technologies, the Federal Energy Management Program, and the Distributed Energy Resources Program. Photovoltaics (PV) research is coordinated with the DOE Office of Science. Interagency coordination between DOE and other Federal agencies (Air Force, Army, NASA, and Navy) is accomplished through the government-sponsored Interagency Advanced Power Group (IAPG). The program supports Department of Interior (DOI) solar efforts in national parks, provides Federal Emergency Management Agency with mobile solar systems that generate power immediately after disasters, helps the Army analyze the benefits of solar water heaters and PV on military housing, collaborates with the Department of Housing and Urban Development educating appraisers about solar technology, and works with the Western Governors Association analyzing the costs and benefits of CSP for relevant States.

Evidence: U.S. Department of Interior Press Release, evaluating renewable energy resources on public lands, February 21, 2003. IAPG website (www.grc.nasa.gov/WWW/IAPG/).

Does the program use strong financial management practices?

Explanation: Each year, EERE develops and maintains a Spend Plan and a Measures spreadsheet that links the Spend Plan to annual and long-term goals and measures for each EERE program. The program reviews quarterly costing reports and weekly project status reports. There is no evidence of erroneous payments or statutory violations.

Evidence: FY 2003 Spend Plan and Measures spreadsheet. Sample quarterly costing report.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The National Association of Public Administrators (NAPA) found dozens of management deficiencies in the program's bureau (the Office of Energy Efficiency and Renewable Energy, or EERE) in a review published in 2000. EERE provided evidence that it addressed some of management deficiencies identified by NAPA, and has prepared a Management Action Plan that will address many of the remaining findings. While a few NAPA recommendations have not been addressed (e.g., that EERE conduct periodic audits to assure that cost-sharing partners actually provide funding they agree to), in general, EERE has taken meaningful steps to address most deficiencies.

Evidence: A Review of the Management in the Office of Energy Efficiency and Renewable Energy (NAPA, 2000). Letter Report from Assistant Secretary Garman to Chairman of the House Subcommittee on Interior and Related Agencies on implementation of NAPA recommendations (July 11, 2001). EERE Management Action Plan (August 2003).

Does the program allocate funds through a competitive, merit-based process, or, if not, does it justify funding methods and document how quality is maintained?

Explanation: The program completed a spreadsheet summarizing the conduct of its R&D in accordance with OMB Circular A-11 definitions. More than 80 percent of program funding goes to national labs, about half of which is subcontracted out, almost entirely competitively. Of the remaining (non-national lab) funding, about half is earmarked, and the balance is largely awarded competitively. The strong reliance on competitive awards ensures program quality. Program efficiency can be improved by reducing funding for subcontracts run by the national labs, and instead having the program run the competitive solicitations directly.

Evidence: FY 2003 Spend Plan. Table showing funding allocations as per OMB Circular A-11 definitions for "Conduct of Research and Development."

Does competition encourage the participation of new/first-time performers through a fair and open application process?

Explanation:  

Evidence:  

Does the program adequately define appropriate termination points and other decision points?

Explanation:  

Evidence:  

If the program includes technology development or construction or operation of a facility, does the program clearly define deliverables and required capability/performance characteristics and appropriate, credible cost and schedule goals?

Explanation:  

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: Progress on long-term goal photovolatics goal appears to be on track. Some targets for reducing the cost of solar water heating in non-freezing climates have been missed in the past, in part due to appropriations below the request level, and in part due to technological difficulties with polymer materials. In a 2000 report, the NAS noted that the photovolatics subprogram has demonstrated effective progress.

Evidence: National Academy of Sciences, "Renewable Power Pathways: A Review of The U.S. Department of Energy's Renewable Energy Programs" (2000).

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program achieved its key annual performance target for producer costs of photovoltaic modules. Appropriations below request level contributed to missed targets in solar hot water heating. (Achievement of targets is in part impacted by budget level.) Advances in polymer materials for solar water heaters also proceeded more slowly than expected. Future targets have been adjusted accordingly.

Evidence:  

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The program identified several activities that would seem to promote efficiency and cost-effectiveness, such as developing electronic collection, storage, management and reporting systems that eliminate historic but unneeded reporting, and integrate performance, planning, fiscal and management data. In 2003, the program also reorganized its three, formerly "stovepiped" activities (photovoltaic, concentrating solar power, solar buildings) into a new, unified "systems-driven approach." The new approach is intended to help prioritize activities in the portfolio by relying on analyses of present and potential markets, technology trade-off studies, and R&D reviews. While the approach is commendable, the program could not provide evidence that these activities have improved efficiency and cost effectiveness.

Evidence: Results of the Systems-Driven Approach to Solar Workshop (December 17-18, 2002). DRAFT Summary Report of the DOE Workshop for a Systems-Driven Approach to Inverter Research and Development (July 2003).

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: The program works closely with industry and State programs to advance the state of the art in solar energy technologies. There are no studies comparing this program to similar programs.

Evidence:  

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: In its analysis of the photovoltaics subprogram (the largest component of the program's portfolio), the National Academy of Sciences noted: "Effective progress in developing low-power, off-grid applications has kept many firms in business and is partly responsible for today's billion dollar industry." NAS gave a poor review of the Concentrating Solar Power Subprogram: "CSP's portfolio is mostly politically driven; and no hard measures have been established for measuring progress or allocating funding." Accordingly, the CSP subprogram was phased out in the FY 2003 and FY2004 Budgets while detailed reviews of the potential for CSP were conducted. The reviews are currently under consideration. NAS did not review the solar buildings subprogram.

Evidence: National Academy of Sciences, "Renewable Power Pathways: A Review of The U.S. Department of Energy's Renewable Energy Programs" (2000).

If the program includes construction of a facility, were program goals achieved within budgeted costs and established schedules?

Explanation:  

Evidence: