Detailed Information on the
Biomass and Biorefinery Systems Assessment

Develop guidance that specifies a consistent framework for analyzing the costs and benefits of research and development investments, and use this information to guide budget decisions.

Developing a resource loaded plan in alignment with its multi-year program plan in order to provide a consistent framework for planning its research, development, and demonstration portfolio, estimate costs of and prioritize activities, which will be used to guide budget decisions.

Develop additional performance measures to assess technical progress in promoting commercial biorefineries that can produce fuels, chemicals, materials, and power from biomass.

Work to reduce the number of Congressional earmarks.

Measure: Modeled cost of a mixed, dilute sugar stream suitable for fermentation to ethanol ($/lb). The sugar stream is produced from biomass residues, not high-value corn kernels (the feedstock currently used for commercial production of ethanol). At current biomass feedstock prices, 15??/lb for a sugar stream is equivalent to $2.75 per gallon of ethanol.

Explanation:The cost of the sugar stream ties directly to the price of ethanol, a substitute for gasoline and key output of a biorefinery. Reduction in the cost of sugars can lead to commercialization of biorefineries that produce fuels (such as ethanol), chemicals, heat, and power from biomass. Thus, the measure addresses the program purpose to use biomass to displace the use of fossil fuels. Feedstock costs, which are assumed for future targets and are outside of the control of the program, directly impact the sugar stream cost. The model and associated results used to demonstrate progress will be validated through external independent review.

Measure: Administrative costs as a percent of total program costs (%).

Explanation:This "overhead rate" measure is not a true efficiency measure but is a meaningful surrogate used for all DOE applied R&D and related programs. The objective is to maintain a reasonable overhead rate for effective operation while ensuring that the vast majority of funds address the program purpose. Administrative costs include all Program Direction and Program Support costs plus costs for supporting activities and analysis funded through programmatic appropriations. The targets and actuals represent corporate figures (i.e., for the entire Office of Energy Efficiency and Renewable Energy) because some EERE Program Direction costs are difficult to parse at the program level in a meaningful way. Appropriation levels for EERE programs and for EERE Program Direction directly affect whether the target is achieved. The baseline and targets for this measure are under development.

Measure: Modeled cost of a cleaned and reformed biomass-derived synthesis gas in dollars per million British thermal units ($/MBtu).

Explanation:Biomass-derived syngas is a substitute for natural gas and a key output of a biorefinery. Reduction in the cost of syngas can lead to commercialization of biorefineries that produce fuels, chemicals, heat, and power from biomass. Thus, the measure addresses the program purpose to use biomass to displace the use of fossil fuels. Feedstock costs, which are assumed for future targets and outside of the control of the program, directly impact syngas cost. Congressional earmarks will prohibit progress in 2006. Past targets and actual data were revised in 2005 based on improved modeling methods. The model and associated results used to demonstrate progress will be validated through external independent review.

Is the program purpose clear?

Explanation: The Biomass Program's purpose is to help displace the use of fossil fuels by conducting research and development aimed at reducing the cost of converting domestic biomass resources into fuels, chemicals, heat and power through the development of biorefineries. The program's mission statement is "to partner with the U.S. industry to foster research and development on advanced technologies that will convert our Nation's biomass resources into affordable industrial products (including energy and higher valued chemicals and materials) through the development of biorefineries."

Evidence: FY 2006 Budget (Mission). Biomass R&D Act P 71-73 (Title III of the Agricultural Risk Protection Act of 2000, P.L. 106-224), June, 2000. Energy Policy Act of 1992, P.L. 107-171.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program funds research and development that can help address technical and cost challenges to increased use of biomass for production of power, fuels, and chemicals. Thus, the Program supports the National Energy Policy goals of increasing domestic energy supplies and increasing America's use of renewable energy. To the extent that the program's research results in increased use of fuels and chemicals produced from domestic biomass, the program can also help to improve energy security by helping to displace use of petroleum, much of which is imported.

Evidence: National Energy Policy, 2001.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program is the major Federal program designed to help develop technology for a biorefinery. Other Federal (primarily USDA) state, and private sector entities conduct similar research, but the program partners with these entities to avoid duplication of effort. In 2000, the Biomass Research and Development Act created a Biomass R&D Board and a Biomass R&D Technical Advisory Committee, both of which help to guide and coordinate Federally-funded biomass R&D. In addition, the Technical Advisory Committee, comprised of industry and academia representatives, ensures that the Federal effort does not duplicate industry's efforts by reviewing DOE's and USDA's annual progress in biomass R&D and making recommendations for future activities. DOE and USDA jointly prepare for the Technical Advisory Committee a summary of DOE- and USDA-sponsored projects and funding by technical category to help ensure efforts are not duplicated.

Evidence: FY 2006 Budget. Biomass R&D Act Part 71-73 (Title III of the Agricultural Risk Protection Act of 2000, P.L. 106-224), June, 2000. USDA and DOE R&D Activities by Roadmap Category, 2004. Biomass Research & Development Website, www.bioproducts-bioenergy.gov

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is structured to directly fund biomass technology development, guided by input obtained from external experts. The program solicits periodic peer and stakeholder reviews of ongoing and planned activities. Reports of these reviews and resolution of feedback demonstrate a high degree of peer and stakeholder support for the program's focus and balance of activities. The current focus is on R&D related to sugars and syngas platforms, products development and integration into biorefineries with planned pilot-scale projects in appropriate areas. Other policy approaches to promote development and deployment of biomass technologies include tax incentives for ethanol production, production tax credits for power produced from biomass, and a renewable fuels standard mandating minimum use of biofuels. There is no evidence that any of these approaches would be more effective or efficient than directly funding biomass R&D.

Evidence: Roadmap for Biomass Technologies, 2002. Roadmap for Agricultural Biomass Feedstock, 2003. Biomass Program Review, Nov. 2003.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: DOE currently estimates the potential benefits of its applied R&D programs, but needs to improve consistency in methodology and assumptions across and within programs to help target resources based on costs and potential benefits. External peer reviews have not assessed whether some program resources may benefit the private sector without producing public benefits or in the absence of a market failure (i.e., lack of incentive for industry investment). The program is working to address these issues. In addition, Congressional earmarking of program funds (roughly 40 percent annually) have severely reduced resources intended for various useful activities. However, it should be noted that the program's annual R&D solicitations are guided by the strategy or specific directives described in multi-year plans, roadmaps, the Biomass Technical Advisory Committee's input, and the Biomass R&D Act of 2000.

Evidence: Multi-Year Technical Plan 2004. Biomass R&D Act of 2000.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program's long-term performance measures relate to production costs of biomass-derived a sugar solution (a precursor to ethanol, a gasoline substitute) and synthetic gas (syngas, a natural gas substitute). Sugars and syngas are the basis for economically producing fuels, chemicals, heat and power from biomass in a "biorefinery". The program has recently developed biorefinery technology pathways, and has established the cost and technical performance measures to be used to validate each pathway. The sugars and syngas cost targets are important indicators within most of the pathways being considered. The program will soon replace the syngas measure with another measure concerning biorefineries. Therefore, targets for the syngas measure do not extend beyond 2007.

Evidence: FY 2006 budget request. DOE Summary Program Plan 2005. Biomass Pathways Analysis. Multi-Year Technical Plan 2004 (being updated).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The target costs of biomass sugars and syngas represent are ambitious but realistic. These cost targets incorporate feedstock costs, capital and O&M costs, spare parts costs, plant on-line factor, etc. and ensure that all meaningful cost factors are adequately accounted for. The models used to generate cost targets and assess actuals ("state-of-technology") were refined for preparation of the FY 2006 Budget. Past targets and actuals were revised accordingly. The program will soon replace the syngas measure with another measure concerning biorefineries. Therefore, targets for the syngas measure do not extend beyond 2007.

Evidence: FY 2006 budget request. DOE Summary Program Plan 2005. Biomass Pathways Analysis. Multi-Year Technical Plan 2004.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program's assesses progress on its two long-term measures annually, and therefore the long-term measures also serve as annual measures. The two measures are the cost of biomass-derived syngas (a substitute for natural gas) and the cost of a dilute sugar stream suitable for producing ethanol (a substitute for gasoline). The syngas measure will soon be replaced by a measure that more directly addresses the program's goal to help industry make a business case for commercializing biorefineries that produce fuels, chemicals, heat, and power from biomass, much like an oil refinery. (The current annual milestones for the program's "Utilization of Platform Outputs" activity are not easily converted to a PART measure). Milestones that the program has used include completion of bench-scale and pilot-scale tests, mid-term cost goals for sugars and syngas, number of bio-based products for which technical and market potential will be established, and evaluation of the costs of biorefinery output using the latest technological parameters.

Evidence: FY 2006 budget request, DOE Summary Program Plan 2005, Biomass Pathways Analysis, Multi-Year Technical Plan, 2004.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The two annual measures are the cost of biomass-derived syngas (a substitute for natural gas) and the cost of a dilute sugar stream suitable for producing ethanol (a substitute for gasoline). Both measures have baselines and ambitious targets. In addition, the program has used milestones to assess progess annually. Examples include completion of bench-scale and pilot-scale tests, mid-term cost goals for sugars and syngas, number of bio-based products for which technical and market potential will be established, and evaluation of the costs of biorefinery output using the latest technological parameters. Additional annual measures that address the program purpose, tie to the long-term goals, and fit the PART format are under development.

Evidence: FY 2006 budget request, DOE Summary Program Plan 2005, Biomass Pathways Analysis, Multi-Year Technical Plan, 2004.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Sample program contract documents indicate that project performers commit to working toward program goals. Projects are subject to semi-annual or annual reviews, and project performers must submit monthly or quarterly status reports. The National Labs are rated annually according to performance in meeting milestones and other requirements under the operating contracts. All research projects are subject to periodic technical reviews and/or the stage-gate review process where the independent reviewers are provided with program goals as background. The program's Systems Integrator position was created in 2004 to ensure that all the program's activities at National Laboratories and partnering organizations serve program goals.

Evidence: Sample program contracts documents. Sample quarterly status reports of project performers. Sample ratings of National Lab research projects.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program plans for programmatic reviews once every two years in accordance with Office of Energy Efficiency and Renewable Energy (EERE) Peer Review Guidelines. The guidelines specify scope of the reviews and criteria for selecting independent evaluators The latest peer review took place in November 2003. In addition, the program conducts annual technical reviews and periodic "stage-gate" reviews of research activities. In stage-gate reviews, independent internal and external reviewers assess whether appropriate milestones and criteria are met before R&D advances to a new level. Over the past few years, the Interagency Biomass R&D Board and Biomass R&D Technical Advisory Committee, a Federal Advisory Committee, have provided useful input through an annual review of USDA and DOE R&D strategies. Approximately 25 private sector experts on the Advisory Committee help ensure that the perspectives of industry, farmers, state governments, and other stakeholders are recognized by the program.

Evidence: EERE Peer Review Guidelines, August 2004. Biomass Research and Development Technical Advisory Committee Recommendations, Dec 2001 and Sep 2002. Biomass Program Review Nov. 2003. Stage-gate reviews including Sugars Platform 2004. Stage-Gate Management in the Biomass Program, 2005.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The budget includes a table that shows funding by Program Goals, of which there is one for the Biomass Program. The Department's Program Goal is not quantitative and differs from the long-term measures included in this assessment. Nevertheless, it accurately reflects the program mission. In addition, the EERE Business Administration Office is able to estimate administrative costs associated with each of EERE's program, including the Biomass Program, and will provide this information with its budget submission.

Evidence: FY 2006 Budget.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The National Academy of Public Administration (NAPA) conducted a management review of all Energy Efficiency and Renewable Energy (EERE) programs in 2000. NAPA determined that for EERE to accomplish is strategic goals, EERE needs to integrate its diverse sectors and technologies into a cohesive organization to make decisions based on priorities and tradeoffs. To address this deficiency, in 2002 EERE conducted a major reorganization of all programs including the Biomass Program. The reorganization combined biomass activities from 3 offices into one to improve the programs' effectiveness by focusing resources on a limited and more coherent set of goals and objectives, reducing overhead expenses, and exploiting synergies among similar activities. Under this new structure, a new Business Administration office was charged with developing guidance on planning for all the programs, and integrating EERE-wide multi-year targets in benefits models. The new Project Management Center monitors technical projects, provides HQ with information on technical progress that is useful to the planning process, and ensures the development of business practices focused on achieving mission objectives, priority service, and accountability. Since 2001, the program has prepared multiyear program plans to help guide planning and management decisions and development of long-term goals. EERE has issued a template multi-year program plan to standardize planning considerations and structure.

Evidence: NAPA, A Review of Management in the Office of Energy Efficiency and Renewable Energy, 2000. Biomass Summary Multi-Year Program Plan, April 2005. EERE Multi-Year Program Plan Template;

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: DOE currently estimates benefits of its applied R&D programs, but needs to improve consistency in the methodology and assumptions it uses to help inform resource allocation decisions. The program cannot currently make meaningful comparisons of its costs and benefits to other programs with similar goals.

Evidence: EERE benefits estimates, available on line at www.eere.energy.gov/office_eere/ba/gpra.html

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: To set priorities, the program uses pathways analysis and an assessment process consisting of three steps. Step 1 - characterize and develop a hierarchy of the technical barriers; Step 2 - identify needed R&D, focusing on areas of possible cost reduction or performance enhancement; Step 3 - set priorities and allocate resources within expected budget. The Multi-Year Program Plan groups all activities according to which technical barrier each activity addresses. The program also developed a process designed specifically to prioritize projects supported by a consortium of national labs, which resulted in a rank ordering of more than 25 specific activities.

Evidence: Biomass Summary Multi-Year Program Plan, April 2005. A Process for Prioritizing Projects in the Biomass Program, December 1, 2003. The program prioritizes its major program activities as follows: 1. Integrated biorefineries 2. Products core R&D 3. Sugar platform core R&D 4. Biomass feedstock interface R&D 5. Thermochemical platform core R&D

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: DOE requires that each program establish and track (on a quarterly basis) progress toward long-term and near-term program performance goals in a Deparmental tracking system called Joule. Some of the goals that the program has been reporting on are process-oriented and differ from goals established in this assessment. Nevertheless, the program has provided regular reporting on progress against established milestones. The program also collects regularly performance data from project performers. Each R&D project or lab research task has established milestones and metrics against a project schedule. These milestones and metrics have been identified and categorized in terms of how they contribute to the meeting the program's long term and annual targets specific to performance measures in key technology areas. Each of these project and research tasks provide quarterly reports to the Project Management Center (which serves all Energy Efficiency and Renewable Energy programs). The PMC documents progress of project performers against milestones and financial performance on a quarterly basis and documents its assessment of project progress. The PMC forwards the quarterly status reports along with the PMC assessments to the HQ program office for their review. In addition, in FY 2005 the program instituted the practice of an internal quarterly meeting to review of all R&D tasks and grantees work with the participation of HQ program staff, the PMC and National Laboratory Relationship Managers. The quaterly written and conference call reviews are used to identify potential issues and redirect resources if necessary. In addition the program conducts periodic "stage-gate" reviews for specific technologies as well as a biennial peer review of the entire program. During these reviews all major performance goals as well as the milestones and metrics that support the major performance goals are reviewed for progress toward their achievement. The results of these reviews are utilized to set program and project priorities and allocate resources to projects within expected budget. The results are also used to identify future R&D focus areas for possible cost reduction or performance enhancement and to determine whether funding should continue for individual projects and lab research tasks.

Evidence: Sample quarterly reports from DOE Joule performance tracking system. Sample quarterly report from the EERE Project Management Center.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The annual performance appraisals of all EERE Program Managers include criteria directly related to cost, schedule, and performance results. EERE reviews these criteria monthly in the EERE Monthly Management Reviews. Many EERE contracts include award fee and other performance criteria to hold those partners accountable.

Evidence: Performance Plan and Performance Appraisal Form for Performance Management System Employees; sample contract documents.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The FY 2005 and 2006 apportionments show that the program had nominal unobligated carryover from previous fiscal years (less than one percent), indicating that nearly all program funds were obligated within the fiscal year. However, in FY 2005, the program's uncosted balances (funding obligated but not yet spent) totaled $91.4 million, relative to a FY 2005 appropriation level of $87.5 million. The high uncosted balances are the reason for the "No" answer. The major reason for the program's high uncosted balances is Congressionally-directed projects, which comprise a significant portion (about 40 percent annually) of the program's budget and account for $67 million (73 percent) of the uncosted balance. Recipients of Congressionally-directed project funds are sometimes unfamiliar with government contracting rules and procedures, causing delays in costing. Despite the "No" response because of uncosted balances, it is noteworthy that the program has procedures to ensure funds are spent for the intended purpose. Specifically, the program develops an Annual Operating Plan, which is reviewed internally to ensure that new funding is planned to be obligated consistent with the appropriated purpose. EERE also develops a Spend Plan for all of its programs. The program uses data from Departmental procurement and financial systems and similar data from National Laboratory partners to ensure that actual expenditures occur for purposes and on a schedule consistent with the Spend Plan.

Evidence: 2005 and 2006 apportionments. FY 2005 report on uncosted balances for the Biomass Program. FY 2005 Annual Operating Plan. FY 2005 Financial Status Report. FY 2005 Spend Plan.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: All EERE programs, including the Biomass Program, have established the EERE corporate overhead rate (i.e., the ratio of administrative costs to total program costs for implementing all EERE programs) as the efficiency measure. Consistently tracking the overhead rate and ensuring that it's within a reasonable range should help to improve efficiency. EERE's reorganization in 2002 clarified lines of responsibility and eliminated organizational "stovepipes" by consolidating planning, budgeting, and analysis into a single business administration office. The reorganization reduced management layers, although staff levels remained the same. EERE developed a suite of new reports and tools to improve program managers' access to EERE cost, obligation, and procurement data. EERE is consolidating several legacy IT systems into a single program management system that will track all required information on a project by project basis (cost share, type of contract according to A-11 definitions, etc.). In addition, EERE is working to phase down support services at National Laboratories for cost-effectiveness reasons.

Evidence: EERE Corporate Planning System (CPS). Presentation on EERE Overhead Efficiency Measure Schedule, December 21, 2004. EERE Reorganization "All Hands" presentation: www.eere.energy.gov/office_eere/pdfs/eere_reorg.pdf. IT Investment Plan (EERE IT Business Case Number 019-20-01-12-01-1011-00-304-101).

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Biomass R&D Act of 2000 provided funds for USDA to conduct R&D similar to work conducted by the Biomass program. To avoid duplication and improve coordination of efforts, the program has issued annual USDA/DOE joint solicitations for biomass R&D activities since FY 2003. In addition, USDA feedstock researchers participated on the March 2005 stage-gate review for feedstock R&D supported by the Biomass program. The program also participates in the Metabolic Engineering Working Group (MWEG), a group of eight federal agencies that support metabolic engineering research. Established through the Office of Science and Technology Policy in 1995, MWEG holds monthly meetings, annual workshops, and occasional seminars to improve coordination among agencies. MWEG has supported joint solicitations and maintains a project inventory.

Evidence: FY03 and FY04 DOE/USDA joint solicitations. Feedstock Infrastructure stage-gate review, March 2005. Metabolic Engineering Working Group Report.

Does the program use strong financial management practices?

Explanation: Many traditional aspects of accounting and financial management occur within the Departmental CFO Headquarters and Field organizations -- outside of the day-to-day control of EERE's programs. DOE had received clean audit opinions with no material weaknesses for several consecutive years until FY 2005, when transition to a new accounting system disclosed some financial weaknesses, which are being addressed corporately. In response to the review of EERE business management by the National Academy for Public Administration (NAPA), EERE initiated a wide range of improvements, including the development and routine maintenance of the EERE Spend Plan; the Corporate Planning System's measures spreadsheet that links the Spend Plan to near- and long-term goals and measures; and periodic EERE-wide reviews of those processes by the EERE Office of Business Administration. It should be noted that in a recent DOE Inspector General report, the IG found that, on the whole, more than half of the 20 EERE cooperative agreements reviewed "did not receive sufficient management attention." The IG also noted that "EERE officials acknowledged weaknesses in project management and initiated action to improve their project management approach." In addition, DOE IG reports from 2001 and 2003 identified management issues with specific Biomass projects. Financial management isses have been address through the development of the EERE Project Management Center, and PMC's instituting standard practices for financial management and control. All awards have additional provisions requiring the recipient to keep DOE aware of their financial status and soundness. The EERE Assistant Secretary issued guidelines in a memo dated January 7, 2002, and the Golden Field Office program manager issued additional guidance for monitoring earmarked funds on January 7, 2004. In addition, project management staff at the Golden Field Office has increased from about 3.5 FTEs (Federal and contractor staff) to 12.5 FTEs to oversee and manage the projects.

Evidence: NAPA, A Review of Management in the Office of Energy Efficiency and Renewable Energy, 2000. DOE IG Report No. 0689, Selected Energy Efficiency and Renewable Energy Projects, 2005. DOE IG Report No 0630, The McNeil Biomass Project, 2003. DOE Report No 0513, Financial Assistance for Biomass-to-Ethanol Projects, 2001. Guidelines on Financial Management, January 7, 2002. Memo from John Kersten, Golden Field Office Manager, Monitoring of Financial Assistance Awards Using Earmarked Funds, January 7, 2004.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The National Association of Public Administrators (NAPA) found dozens of management deficiencies in the program's bureau (EERE) in a review published in 2000. EERE provided evidence that it addressed some of management deficiencies identified by NAPA, and has prepared a Management Action Plan to address many of the remaining findings. While a few NAPA recommendations have not been addressed (e.g., that EERE conduct periodic audits to assure that cost-sharing partners actually provide funding they agree to), in general, EERE has taken meaningful steps to address most deficiencies. A follow NAPA report indicates that EERE has addressed or begun to address 47 of 54 management recommendations.

Evidence: NAPA, A Review of Management in the Office of Energy Efficiency and Renewable Energy, 2000. NAPA, Office of Energy Efficiency and Renewable Energy: Reorganizing for Results, 2004.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The program's appropriations are heavily earmarked each year (approximately 40 percent of program funding over the past several years). Because of the earmarks, less than 90 percent of program funding is awarded competitively, and therefore the program must receive a "No" according to PART guidance. However, for funds available, the program follows the DOE policy that all discretionary financial assistance, competitive or noncompetitive, be awarded through a merit-based selection process. Independent, technically qualified reviewers use clear and appropriate pre-defined criteria during the merit review. All grantee applications are subject to a merit review.

Evidence: Office of Energy Efficiency and Renewable Energy, Objective Merit Review of Discretionary Financial Assistance and Unsolicited Proposals, Federal Register Notice December 20, 2001, P. 65686.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program follows DOE's Guide to Financial Assistance (March 2005). Each grantee is required to provide regular technical and financial status reports. For most grantees, the status reports are due quarterly. For a very few of the congressional-directed projects and fundamental R&D projects, status reports are required semi-annually. EERE's Project Management Center (PMC) reviews and documents its assessment of progress and financial performance. The PMC forwards the status reports to the HQ program office for their review on a quarterly basis. PMC project officers regularly conduct site visits, teleconferences and review meetings with the grantees. Further, in FY 2005 the program instituted an internal quarterly review of all R&D tasks and grantees' work with the participation of PMC and HQ program staff, and National Laboratory Relationship Managers. These procedures address shortfalls in oversight practices identified in previous DOE Inspector General reports on projects managed by the Biomass Program

Evidence: DOE's Guide to Financial Assistance, March 2005. Biomass Program Quarterly Reports compendium. DOE IG Report No 0630, The McNeil Biomass Project, 2003. DOE Report No 0513, Financial Assistance for Biomass-to-Ethanol Projects, 2001.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program collects grantee performance data on a quarterly basis and a final Scientific/Technical report at the end of each project. The performance data is made available via project fact sheets that are published annually on the program's website. The final reports are publicly available through the Office of Scientific and Technical Information. In addition, biennial Program Peer Reviews are open to the public and publicized in advance (e.g., the November 2003 review and the planned November 2005 review). Further, Stage-gate Reviews are held on a regular basis to evaluate the status of the R&D tasks and grant projects within specific focus areas. The results of both types of reviews are posted on the program's website.

Evidence: Biomass Project Fact Sheets. 2004 Peer Review. Stage-gate Reviews in 2004 and 2005.

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: The program reports the following allocation of its $92 million in FY 2004 funding: $21 million (23 percent) awarded competitively, $29 million (31 percent) allocated to Department of Energy National Laboratories, and $42 million (46 percent) awarded in non-competitive contracts, almost all of which represent Congressional earmarks. Previous years have similar allocations. While earmarks are outside of the control of the program, the extent of earmarks affects program quality and the ability of the program to achieve its goals; this is the basis for a "No" response. However, for funds within its control, the program maintatins quality by making competitive awards or allocating funds to five National Labs with unique capabilities. The program should continue to ensure that funding for national labs is used only for work that cannot be conducted by universities or the private sector.

Evidence: Funding allocation spreadsheet showing funding awarded competitively, non-competitively, and to National Labs. Unique cababilities of national labs: National Renewable Energy Lab (NREL) is the world leader in the area of lignocellulosic conversion. Industry partners utilize NREL's surface characterization studies, pretreatment advances, and design simulation to catalyze economic cellulosic conversion. They also make use of NREL's life cycle analysis capabilities to determine the long-term direction and viability of their selected projects. Pacific Northwest National Lab (PNNL) has a unique capability in combinatorial catalysis, which allows it to screen large number of catalysts in a very short period of time. This capability does not reside broadly in industry for high pressure and temperature catalyst discovery. PNNL is also a leader in fungal genomics. Argonne National Lab (ANL) has unique expertise in separative fermentation technologies and in analysis models to predict the effects of biofuels on the environment and on energy sustainability. Idaho National Lab (INL) has unique expertise in the development of novel harvesting methods and equipment, as well as storage and delivery infrastructure systems, which can help to lower future biomass feedstock costs. Oak Ridge National Lab (ORNL) has unique expertise in the identification and assessments of biomass resources and their sustainability. ORNL has developed sophisticated models showing the interactive relationships between land utilization, feedstock prices, crop substitution, agronomic practices, and issues of sustainability.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has two key long-term measures: the cost of biomass-derived syngas (a substitute for natural gas) and the cost of a dilute sugar stream suitable for producing ethanol (a substitute for gasoline). The program does not have a long history of demonstrated performance against targets for these measures. Also, the syngas measure will soon be replaced by a measure that more directly addresses the program's goal to help industry make a business case for commercializing biorefineries that produce fuels, chemicals, heat, and power from biomass, much like an oil refinery. The program has missed some targets because Congressional earmarks have historically tied up about 40 percent of the program's budget each year. However, peer reviews indicate that projects within the control of the program are generally achieving their technological milestones.

Evidence: FY 2006 budget request. Biomass Program Peer Review, Summary of Results, November 2003

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program's assesses progress on its two long-term measures annually, and therefore the long-term measures also serve as annual measures. The two measures are the cost of biomass-derived syngas (a substitute for natural gas) and the cost of a dilute sugar stream suitable for producing ethanol (a substitute for gasoline). The program does not have a long history of demonstrated performance against targets for these measures. Also, the syngas measure will soon be replaced by a measure that more directly addresses the program's goal to help industry make a business case for commercializing biorefineries that produce fuels, chemicals, heat, and power from biomass, much like an oil refinery. The program has missed targets because Congressional earmarks have historically tied up about 40 percent of the program's budget each year. However, peer reviews indicate that projects within the control of the program are generally achieving their technological milestones. The program is developing additional annual performance measures appropriate for tracking overall progress of the program toward the long-term goal of commercializing biorefineries to help displace use of fossil fuels.

Evidence: FY 2006 budget request. Biomass Program Peer Review, Summary of Results, November 2003.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: All EERE programs, including the Biomass Program, have established the EERE corporate overhead rate (i.e., the ratio of administrative costs to total program costs forimplementing all EERE programs) as the efficiency measure. A Yes response requires that the program demonstrate improved efficiency or cost effectiveness over the prior year. EERE established a baseline in 2004, and set a corporate target for FY 2006 and beyond. However, there is no history of actual performance and therefore no demonstration of whether efficiency over the prior year has improved or remained within an acceptable range. The "Small Extent" rating reflects the fact that EERE has examined closely the use of national laboratories to conduct administrative functions, and has begun to transition some of this work to the private sector and EERE's Project Management Center. EERE projects savings of $4 million in 2005 alone, although there is no independent verification of this savings or of improved administrative efficiency.

Evidence: EERE, "Transition of Non-R&D National Laboratory Funding to the Private Sector and PMC, Final Report and Interim Plan," April 2005.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The program has no counterpart in the Federal sector. USDA invests a much smaller amount in R&D that complements the program's activities such as improvement of energy crops and a number of starch-to-fuels conversion research projects. The USDA activities are not a program that is comparable to the Biomass Program.

Evidence: USDA and DOE R&D Activities by Roadmap Category, 2004; Material from USDA/DOE feedstock stage-gate review of March 2005.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: In November 2003, the program was reviewed by independent experts with respect to goals of each project, alignment of activities with Biomass program goals, whether the project is appropriately focused on barriers, whether R&D milestones were met, path to overcome impediments if milestones were not met, adequacy of resources for each activity, etc. The reviewers generally felt that most aspects of the program were on track. Average ratings in six categories for each of five major program activities ranged from 2.42 to 3.56 on a 0 to 4 scale. Further, stage-gate reviews are held on a regular basis to evaluate the status of the R&D tasks and grant projects within specific focus areas to determine whether technologies are ready for the next stage of development. The Feedstock portfolio stage-gate review provided mixed reviews; some projects were on track, some were not, and some were apparently misplaced analysis projects that should not have been included in the review.

Evidence: Biomass Program Review, November 2003. Stage-gate Review, March 2005.