Detailed Information on the
Electric System Research and Development Assessment

Continue to work with Congress to reduce earmarking

Develop a methodology for estimates of costs and benefits that are comparable across all DOE applied research and development programs, emphasizing common assumptions and a consistent approach to incorporation of risk, and demonstrating use of this information in budget decisions.

Measure: High Temperature Superconductivity - By 2020, develop prototype wire achieving 1,000,000 length-critical current (A-m) for second generation wire

Explanation:High Temperature Superconductivity - HTS Wire

Measure: High Temperature Superconductivity - Maintain progress in achieving increasingly powerful coils for electric power applications such as transformers and motors: magnetic field (Tesla) produced by test coil at 65K.

Explanation:High Temperature Superconductivity - HTS Coils

Measure: Visualization & Control - By 2014, develop tools and algorithms to enable an automatic, smart, real-time switchable network for transmission system operations that enables secure and reliable grid operations for major regions of the grid that is hardened against cyber attacks. Definitions for Target: PMUs - phasor measurement unit; dv - distribution voltage

Explanation:The Visualization and Controls program will develop technologies (tools and algorithms) that will: 1) Improve the response time of the Transmission System to system disturbances to reduce the number and spread of outages; 2) Reduce the operating margins by allowing the system to operate closer to its limits.

Measure: Distributed Energy Resources - Demonstrate peak load reduction on distribution feeders with the implementation of Distributed Energy (DE) and Energy Management (EMS) at a cost competitive with a system/capacity upgrade (i.e. cost not to exceed $1,600 per kW in 2001 dollars). Measured in Percent (%) Reduction in Peak Load and Number of Feeders Analyzed/Demonstrated.

Explanation:Distributed Energy Resources - Renewables and Distributed Systems Integration

Measure: Maintain an R&D program direction to R&D program funding ratio of less than 12%.

Explanation:Program direction includes salaries and benefits, travel, support services, other related expenses, communications and outreach, and project management in the field. Project management in the field includes efforts that support or contribute to improved organization of program management, logistics management, project monitoring and reporting, information technology, data collection, budgeting, accounting, performance auditing, and administrative technical support for conferences and training programs. This activity does not include technical support services that are used by line staff to implement the programs.

Is the program purpose clear?

Explanation: The mission of the Office of Electricity Delivery and Energy Reliability Office Research and Development Program is to increase the reliability, efficiency, and environmental integrity of the nation's electrical transmission and distribution systems. This is accomplished through a comprehensive research, development and demonstration program which focuses on: 1) strengthening grid stability and reducing the number and duration of operational disturbances (reliability); 2) increasing energy efficiency of the electric delivery system through reduced energy losses (energy efficiency); 3) reducing peak prices and price volatility of electricity, increasing asset utilization (capacity factor of Transmission and Distribution (T&D)), and improving accessibility to a variety of energy sources for electricity generation (system efficiency); and 4) hardening energy infrastructure so it can detect, prevent, and mitigate external disruptions to the energy sector (security).

Evidence: 1. EPACT 2005 Section 921- 925 2. National HTS Competitiveness Act PL 100-697 (1988) 3. EPACT 1992, PL 102-485 4. National Transmission Grid Study 5. National Energy Policy

Does the program address a specific and existing problem, interest, or need?

Explanation: The electrical delivery system is integral to the economic and national security of the U.S. Market uncertainties (restructuring), aging infrastructure, and lack of real-time information are contributing to shortcomings in electric capacity, vulnerabilities in security, and threats to reliability. The current transmission and distribution system is limited by the speed in which it can respond to disturbances. This increases vulnerability to higher number and greater spread of long-term outages. The 2003 Blackout, which affected 8 states and Ontario, Canada, with estimated economic consequences of $4-10 billion, illustrates this. Since 1996 industry funded long term R & D has been all but eliminated. (Electric Power Research Institute's (EPRI) budget has declined 54% in this time.) This is because R & D funding is no longer captured in the regulated rate base but comes out of shareholder earnings. Additionally, with restructuring, benefits of R & D cross jurisdictional boundaries of utilities, regional transmission organizations (RTOs), and independent systems operators.

Evidence: 1. National Transmission Grid Study 2. National Energy Policy 3. EPACT 2005 Section (DOE website) EPACT 2005 Sections [368(a); 368(b); 921; 922; 923; 924(b); 925(a); 925(b); 925(d); 1101(b); 1106; 16 USC 824 Sec. 216 (a)(1), (a)(2), h(7) (A), (h)(7) (B); 1234; 1252(d); 1816(b);1822;1832(c);1839] 4. Final Report on the August 14, 2003 Blackout 5. LBL Understanding the Cost of Power Interruptions to U.S. Consumers 6. Steps to Establish a Real-Time Transmission Monitoring System for Transmission Owners and Operators (FERC/DOE, 2006)

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: OE funds technology R & D that no other organizations funds. DOE's R & D address issues which cross state and regional jurisdictional boundaries. OE coordinates with other affected Federal, state, and private-sector entities. For example, OE works with the California Energy Commission (CEC), the New York State Energy Research and Development Authority (NYSERDA), the Electric Power Research Institute (EPRI), DoD, DHS, other DOE program offices (FE, EERE) to develop joint strategies. Since many of the benefits from OE's R and D are public (national economy, national security) and can not be captured by private industry, and because of regulatory uncertainty the private sector (and public power) are unwilling to make R & D investments without significant federal participation.

Evidence: 1. Portfolio Assessment??FY04 Results OETD & PIER Programs, October 2004, Navigant Consulting 2. Merit Review Panels and Individual Project Peer Reviews (sensitive information, internal use only): i) HTS Peer Reviewers, ii) Energy Storage Peer Reviewers, iii) Distribution Transformation Peer Reviewers, iv) Transmission Reliability Peer Reviewers, 2005, (for 2004 report go to: http://www.oe.energy.gov/transre4.cfm) v) Distributed Energy Peer Reviewers (http://www.energetics.com/depeerreview05/agenda.html) 3. EERE/OE Joint Workshop on Energy Storage: Workshop Notes 4. Title III: DOE-DoD Collaboration - http://www.acq.osd.mil/ott/dpatitle3/, project description.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that a different approach or mechanism would be more effective or efficient. While electricity transmission is a regulated monopoly, reliability hasn't been regulated. Recent statutory changes have established a mechanism to establish mandatory reliability standards; however it isn't clear that in the mid-term this will have meaningful impact on reliability. Historically, economic regulators have not allowed for technology development and precluded or delayed the introduction of newer technologies that could offer greater reliability, efficiency, and capability. Neither the market nor regulators currently support R & D investments that will ensure the future reliability and security of the national electricity delivery system.

Evidence: 1. Individual Project Peer Reviews (sensitive information, internal use only): i) HTS Peer Review, ii) Energy Storage Peer Review, iii) Distribution Transformation Peer Review, iv) Transmission Reliability Peer Review, 2005, (for 2004 report go to: http://www.oe.energy.gov/transre4.cfm) v) Distributed Energy Peer Reviews (http://www.energetics.com/depeerreview05/agenda.html) 2. National Electric Delivery Technology Roadmap 3. Electric Light and Power article on utility technology/R&D investment following blackout 4. Smartgrid Article 5. Transforming the Electric Infrastructure 6. Western Area Power Administration Budget Testimony 7. Control System Security Roadmap Briefing

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: DOE currently estimates the potential benefits of its applied R&D programs, but needs to improve consistency in methodology and assumptions across and within programs to help target resources based on costs and potential benefits. The Department is working to address this issue. Given the national interests at stake, both security and economic, the level of subsidy to electricity consumers is warranted. The ultimate beneficiaries of the program are the citizens of the U.S. To the extent that the program is earmarked by Congress, program funds may be inefficiently targeted.

Evidence:

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has established specific, ambitious long-term performance goals focused on improving reliability and efficiency (environmental and economic). It is beyond the program's ability to control electricity demand or operation of the transmission grid, since DOE does not own or operate it. Therefore the program's measures are output measures, but if accomplished, they will contribute to enhanced electric system reliability and efficiency. (Owners of the transmission grid will have to invest in system upgrades in order for reliability and efficiency to improve.) For example, a High Temperature Superconducting metric focuses on developing longer lengths of wire with improved carrying capabilities that will enable equipment that will make the electric system more reliable and efficient (via higher carrying capacities in constrained areas and less electricity losses). The Visualization and Controls metric, development of tools and algorithms enabling an automatic, smart, real-time switchable transmission network contributes to a transmission system that would have increased reliability (avoidance or isolation of system problems) and efficiency (less congestion.)

Evidence: 1. FY 2007 Congressional Budget Request 2. PART Section 4 Program Performance Measures 3. FY 2007- FY2011 OE FYP 4. DOE Strategic Plan Goals

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has established ambitious targets and timeframes for its long-term measures. OE is leading the transition from predictive modeling to sensing performance characteristics (diagnostic cuing) to an automatic real-time response to make the system more reliable and efficient. Slow mechanical breakers transition to fast solid-state switches at high voltages, and heavy copper wire with high losses are replaced by superconducting cables with significantly reduced losses. The performance measure targets and timeframes are achievable, but ambitious. The Program is also enabling energy-consuming products and processes to respond to market prices of electricity and to address the imbalance between electricity supply and demand, to ensure system reliability.

Evidence: 1. PART Section 4 Program Performance Measures 2. CERTS Phasor Technology Roadmap 3. FY 2007- FY2011 OMB Five Year Plan 4. R&D 100 Awards 5. DOE Strategic Plan Goals

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has four discrete, quantifiable, annual output measures that document progress towards the long-term goals and outcomes (reliability, system efficiency, energy efficiency and security) . The annual measure for superconductivity is the length of wire achieved with the highest current current capability (amps-meter) per year. Both length and current can be physically measured on an annual basis. With additional capacity and reduced losses, overall efficiency of the system will increase (energy efficiency) The annual measure for visualization and controls documents progress by the completion of a measurement tool for monitoring the electric system on a yearly basis. For example, one voltage monitoring tool will be completed for grid testing by 2009. With real-time measurement tools for the grid, operations can be automated resulting in an increase in system efficiency, security and reliability. For distributed integration, the annual measure is quantified by measuring the percent reduction compared to a measured baseline on an annual basis. This will result in better asset utilization and system efficiency Finally, for power electronics the measure is based on switching speed (milliseconds). This will be tracked on an annual basis; reducing the switching time. These power electronics will improve the reaction time on the grid thus improving reliability. As for management, the efficiency measure is to maintain total Research and Development Program Direction costs in the range of 8%-12% of total program costs, to demonstrate efficient and effective business and technical support to mission direct programs. The Overhead Efficiency in FY06 was 5.74% and in FY07 it is 7.83%.

Evidence: 1. FY 2007 Congressional Budget Request 2. PART Section 4 Program Performance Measures 3. FY2007-FY2011 OE FYP 4. OE Long Term Goals Paper

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has established baselines and ambitious targets for all annual measures. These baselines were established in consultation with industry experts as part of "Grid 2030 Vision" and the National Electric Delivery Technologies Roadmap.

Evidence: 1. PART Section 4 Program Performance Measures 2. FY 2007 Congressional Budget Request 3. FY2007-FY2011 OE FYP 4. National Electric Delivery Technology Roadmap 5. Grid 2030 Vision Document

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Solicitations are based on performance targets. The selection of awardees is based on applicants' demonstration of how they will contribute to these targets. Contractual performance metrics are based on OE's annual performance targets. For example, solicitation DE-PS02-05CH11270 for distributed systems integration based its contracts on performance targets. All private partnership activities are conducted with commitments from all participants to work to achieve the common vision and milestones. With respect to national laboratories, all field work proposals contain performance targets from the programs. Performance is reported and measured on a regular basis. Industry partners are contractually obligated to report technical progress on a quarterly basis. All contracts include a number of milestones against which progress can be measured. Laboratories also submit quarterly progress reports, measuring against milestone in their Annual Operating Plans and their M&O contracts are evaluated against performance every year.

Evidence: 1. DE-PS02-05CH11270 Contract and Solicitation for Distribution Integration 2. ORNL Distributed Energy Annual Operating Plan, FY2006 3. Southern Company Award and Proposal: Development of an IDMS (This material is corporate sensitive: internal use only) 4. Infrastructure Security Technology FY07 Over Target Request

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Projects and program elements are peer reviewed by an independent, unbiased panel of experts, for relevance and performance. Peer review findings are used to refocus existing efforts. Peer reviews occur at a minimum of every other year on a recurring basis. For example, the findings from the Electric Distribution Transformation Program resulted in the termination of two projects in FY 2005. In addition to individual project reviews, OE conducts an internal portfolio review every other year and the entire portfolio is assessed by independent reviewers in one every two years on a recurring basis. This helps ensure an appropriate balance in projects to meet overall goals. In combination, the various review groups provide a breadth of perspective that ensures the program effectively meets objectives and is relevant.

Evidence: 1. Individual Project Peer Reviews (sensitive information, internal use only): i) HTS Peer Review, ii) Energy Storage Peer Review, iii) Distribution Transformation Peer Review, iv) Transmission Reliability Peer Review, 2005, (for 2004 report go to: http://www.oe.energy.gov/transre4.cfm) v) Distributed Energy Peer Reviewers (http://www.energetics.com/depeerreview05/agenda.html) 2. Portfolio Assessment??FY04 Results OETD & PIER Programs, October 2004, Navigant Consulting 3. DE-PS02-05CH11270 Contract and Solicitation for Distribution Integration

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget Requests explicitly tie resource levels to accomplishment of annual and long-term performance goals, in a complete and transparent manner. Within the budget justification, the R&D program is broken down into both subprograms and activities, allowing the effect of additional or reduced funding to be more easily identified at a detailed level. However, the program does not report full direct and indirect costs.

Evidence: 1. February Monthly Performance Report 2. R&D Scorecard 3. FY 2007 Congressional Budget Request

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has realigned its program structure to eliminate overlap resulting in a more focused program. For example, the complementary Gridwise and Gridworks activities have been merged into the Visualization & Controls budget line. Gridwise and Gridworks were created to target specific needs for developing a smart, self-healing grid and developing next generation utility equipment, respectively. From inception, there was confusion in industry and government about whether these programs overlapped, were redundant or were adequately focused. The two programs were not well understood or supported and DOE was criticized for allowing program overlap between these activities and with on-going work in transmission reliability and distributed energy. To reduce confusion and driving for clearer, more focused research, the program elements were examined internally in DOE and externally with industry. The best elements of all the programs in OE were incorporated into four new program lines designed to rationalize and refocus the program.

Evidence: 1. FY 2007 Congressional Budget Request 2. FY 2006 Congressional Budget 3. OE Realignment and Staffing Memos

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: DOE currently estimates benefits of its applied R&D programs, but needs to improve consistency in the methodology and assumptions it uses to help inform resource allocation decisions. The program cannot currently make meaningful comparisons of its costs and benefits to other programs with similar goals. OE continues to move forward in this regard. Although NEMS does not adequately represent the potential benefits of the electricity infrastructure as it does to other programs, the program is considering other power flow models to enhance NEMS modeling.

Evidence: 1. OE R&D Benefits Estimate Update 2. LBNL White paper on Modeling Benefits of Transmission Upgrades in NEMS

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The Office develops internal priority-ranking tables for each budget development cycle. From a strategic planning perspective, the program involves stakeholders, including subject matter experts, at workshops to provide priority ranking of activities in the roadmap process to define major technical priorities and critical hurdles. For example, in FY 2005, OE determined that the more than $20 million required to complete the construction and operation of the Energy Reliability and Efficiency Laboratory (EREL) could be more effectively spent on other projects. OE terminated the EREL project, and utilized residual funding to support less expensive and more resource-efficient activities under Visualization and Controls.

Evidence: 1. FY07 At Target Priority Table 2. Portfolio Assessment??FY04 Results OETD & PIER Programs, October 2004, Navigant Consulting 3. Example of HTS research projects ended/refocused 4. Distribution Transformation Termination Letter 5. GridWorks Multiyear Plan

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: OE tracks (via the JOULE system) on a quarterly basis progress toward long-term and near-term program performance goals. OE tracks and reports on each major project's cost, schedule and performance on a monthly basis for OE's senior management. When cost, schedule or performance problems arise, such as an unachieved milestone, activity managers and/or project partners take corrective action. For Joule milestones and annual targets, action plans are developed to delineate corrective actions and follow-up action is required to fully address performance concerns and targets. As a result of these processes, the HTS program made a major shift in emphasis to second generation wire when it was apparent that first generation wire research would not meet long-term performance expectations.

Evidence: 1.OE Monthly Performance Financial Reports 2. HTS Monthly Detailed Financial Report Example 3. Storage Financial Monthly Report 4. Storage Monthly Financial Status Report 5. Joule and PMA Scorecard 6. Joule Quarterly Report 7. ORNL Distributed Energy Annual Operating Plan, FY2006 8. Individual Project Peer Reviews (sensitive information, internal use only): i) HTS Peer Review, ii) Energy Storage Peer Review, iii) Distribution Transformation Peer Review, iv) Transmission Reliability Peer Review, 2005, (for 2004 report go to: http://www.oe.energy.gov/transre4.cfm) v) Distributed Energy Peer Reviewers (http://www.energetics.com/depeerreview05/agenda.html) 9. Corporate Planning System (CPS) Report Screenshot 10. Corporate Planning System Milestone Report 11. Portfolio Assessment??FY04 Results OETD & PIER Programs, October 2004, Navigant Consulting 12. Spend Plan Example 13. February Monthly Performance Report with Spend Plan Information 14. NETL Spend Plan Actual and Projected Information 15. Memoranda on standardized quarterly reporting

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: OE managers and program partners are rated and rewarded based on results in achieving annual and longer-term goals per contract language and performance standards. Each level of management is responsible for taking remedial action to address problems with cost, schedule and performance based on the results of peer reviews, programmatic reviews, and monthly or quarterly financial and performance reports and reviews. Program partners that are not performing can have their award fee reduced. M&O Contractors (National Laboratories) are reviewed every year on their performance and award fees can be reduced due to poor performance. Support contracts are reviewed at the end of each performance period and can have their award fees reduced for unsatisfactory performance. Work performed on contracts, grants, and cooperative agreements can be penalized for poor performance; currently use of budget periods in grants and cooperative agreements can be used as off ramps for less than satisfactory performance. However, such a penalty has never been warranted within the program.

Evidence: 1. Performance Plan and Performance Appraisal Form (Federal Employee) 2. M&O Performance Appraisal (Laboratory) 3. DE-PS02-05CH11270 Contract and Solicitation for Distribution Integration (Industry) 4. Memoranda on standardized quarterly reporting

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: OE Spend Plans are reviewed each month with monthly financial performance reports tracking unobligated funds, uncosted obligations, and costed funds on a monthly basis and compare them to projections. The reports detail OE funds by Division to highlight particular activities that are experiencing difficulties in obligating and contracting funds, such as grant programs for universities administered by third parties like NSF that have a different pattern of obligation and costing than other research investments. Corrective action is taken each month as needed to help ensure effective obligation and costing of funds. Program awards are made in a timely fashion. Historically the program has experienced difficulty for a limited subset of new earmarks, in getting funds to earmarked receipients since Congress has been slow to indicate to DOE who the intended receipient of the earmark was.

Evidence: 1. OE Monthly Performance Financial Report 2. Spend Plan Example

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: OE utilizes competitive contract awards whenever possible for both programmatic and program support activities that are discretionary (i.e. non-Congressionally directed) in order to achieve efficiencies and cost effectiveness, including an overhead efficiency of less than 12% for R&D (in accord with DOE's definition for the R&D overhead efficiency measure). OE's program support contractor was selected through a competitive bidding process in accord with the Federal Acquisition Regulations (FAR). In cases where it is cost effective and appropriate, OE utilizes pre-existing contracts that have been competitively awarded or competitively sourced by another program office, the CFO, the OCIO and/or DOE. For example, OE's IT support comes from an OCIO competitively awarded firm (under FAR) and an OCIO competitively sourced firm (under OMB's Circular No. A-76).

Evidence: 1. OE Efficiency Measure 2. Competitive Contract Award Sourcing Policy 3. Support contract solicitation 4. IRM Strategic Plan 5. OE Calendar application (print-off a screen to measure efficiency) (check with Lumas) 6. Corporate Planning System (CPS) Report Screenshot 7. E-Gov Scorecard

Does the program collaborate and coordinate effectively with related programs?

Explanation: OE regularly collaborates with other Federal, State, local and private programs. HTS coordinates with DOD to share information and coordinate research, including a cost-shared program through the Defense Production Act Title III, Yttrium Barium Copper Oxide (YBCO) High Temperature Superconductor Project. Energy Storage participates in joint workshops with the FreedomCAR and Vehicle Technologies Program, and the Solar Energy Technologies Program to discuss common interests and to identify appropriate research activities. OE collaborates with the California Energy Commission on portfolio analysis, as well as work in transmission, distribution, and storage. Similarly, OE coordinates with NYSERDA, EPRI, NERC and other institutions engaged in electricity research.

Evidence: 1. Merit Review Panels and Individual Project Peer Reviews (sensitive information, internal use only): i) HTS Peer Reviewers, ii) Energy Storage Peer Reviewers, iii) Distribution Transformation Peer Reviewers, iv) Transmission Reliability Peer Reviewers, 2005, (for 2004 report go to: http://www.oe.energy.gov/transre4.cfm) v) Distributed Energy Peer Reviewers (http://www.energetics.com/depeerreview05/agenda.html) 2. Portfolio Assessment??FY04 Results OETD & PIER Programs, October 2004, Navigant Consulting 3. National Transmission Grid Study 4. Interagency Agreements with Office of Naval Research 5. HTS Interagency Agreements with AFRL 6. HTS-DoD Title III Partnership 7. Interagency Agreement with EPA 8. Micro-grid CEC&OE Joint Roadmap 9. Microgrids Research Assessment Phase II

Does the program use strong financial management practices?

Explanation: OE's financial management, statements and reports have been in full accord with DOE and Federal requirements. On an annual basis, OE submits to the Secretary of Energy a Management Control and Financial Management System Review, in accordance with the Federal Managers' Financial Integrity Act, which includes consideration of the results of internal management reviews, assurances from major contractors under OE's cognizance, and all other known information. OE reviews STARS information regularly and provides it to research managers to compare to their own project financial information and information provided by labs and research performers in order to identify deficiencies and problems. A 2005 IG investigation of OE's financial management practices was terminated in its early stages, with the investigators determining that there was no evidence found to warrant continuation of its investigation.

Evidence: 1. OE Monthly Performance Financial Report, taken from IDW 2. Support Service Contractor Monthly Reports 3. PMA Scorecard Budget and Performance Integration (Green on the PMA Scorecard) and Scorecard IT Capital Planning & Investment Controls (or CPIC; Green on the PMA Scorecard) 4. Consolidated Quarterly Performance Report (quarterly and annually) 5. Corporate Planning System (CPS) Report Screenshot 6. Monthly detailed reports from IDW 7. OETD Financial Obligations included in Monthly Performance Reports

Has the program taken meaningful steps to address its management deficiencies?

Explanation: OE is implementing a strategic Human Capital Management Plan that will improve office performance with recruiting, training, and succession planning to address management weaknesses. For example, half of the Technology Managers are currently eligible for retirement. To address this succession planning need, OE is adopting a teaming approach for program management, investigating modification of the Department's mentoring program for use in a small office structure, and recruiting Presidential Fellows. Additionally, the DE Program was merged with OE to build upon synergies in research expertise. On the financial side, in FY2005, OE's senior management began reviewing financial and performance reports on a monthly basis in order to take more timely corrective actions when necessary in executing spend plans. Project cost, schedule and performance information will improve further as OE moves to a more automated system of reporting in the final quarter of FY 2006.

Evidence: 1. Human Capital Management Plan 2. OE Staffing Plan 3. OE Realignment and Staffing Memos

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: OE utilizes competitive contracts in both programmatic and program support activities. Outreach encourages participation of new grantees, and selections are made in a fair and open competitive process. However, over 40% of appropriated R&D funding within OE supports Congressionally-directed activities.

Evidence: 1. DE-PS02-05CH11270 Contract and Solicitation for Distribution Integration

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: All projects, including earmarks, have periodic reporting requirements to document grantees' use of funds, and verify that funds are used for their designated purpose and evaluate progress toward goals and milestones. Project managers and field personnel conduct site visits on a regular basis. Most industry partners are contractually obligated to report technical and financial progress on a quarterly basis and supply an annual report. All contracts include a number of milestones against which progress can be measured. Laboratories also submit financial and technical progress reports, measuring against milestone in their Annual Operating Plans and their M&O contracts are evaluated against performance every year. Both labs and industry partners present their work in semi annual peer reviews. Financial data is submitted monthly to DOE's Standard Accounting and Reporting System.

Evidence: 1. DE-PS02-05CH11270 Contract and Solicitation for Distribution Integration 2. HTS-DoD Title III Partnership

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Annual public peer reviews provide an opportunity for program participants, including earmark recipients, to present project performance information to the public in a transparent and meaningful way. Presentations are then posted on the web for public dissemination.

Evidence: 1. EERE Peer Review Guide, August 2004

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: The OE program is heavily earmarked. (FY 2006 46%.) In areas that are not earmarked OE allocates funds and uses management processes that maintain program quality. OE managers analyze portfolios annually focusing on years to commercialization, public benefits, total federal costs, technical risk, and/or cost sharing. Activity managers utilize competitive solicitations and merit criteria to determine how best to allocate their limited resources. Annual peer reviews, and monthly to quarterly tracking of cost, schedule and performance, help activity managers ensure that their projects are performing as expected, and to take corrective action when needed. The percentage of the program competitively awarded is 52% (including National Laboratory in house work). Percentage would be higher without congressionally directed activities. Cost share minimum for competitively awarded projects is 20% based on EPACT 1992. Percentage increases with maturity of technology up to a minimum of 50% for demonstration projects.

Evidence: 1. DE-PS02-05CH11270 Contract and Solicitation for Distribution Integration 2. A-11 R&D Budget Table

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: OE evaluates progress towards the long-term performance goals through its Peer Reviews. The Superconductivity Program is achieving its long terms goals and its technical targets, making the US a world leader in the High Temperature Superconductivity technology. Additionally, the DE Program has consistently met its long-term targets of 2000 and 2006.

Evidence: 1. FY 2007 Congressional Budget Request 2. Individual Project Peer Reviews (sensitive information, internal use only): i) HTS Peer Review, ii) Energy Storage Peer Review, iii) Distribution Transformation Peer Review, iv) Transmission Reliability Peer Review, 2005, (for 2004 report go to: http://www.oe.energy.gov/transre4.cfm) v) Distributed Energy Peer Reviewers (http://www.energetics.com/depeerreview05/agenda.html)

Does the program (including program partners) achieve its annual performance goals?

Explanation: As in FY 2005, the OE Program has had aggressive milestones. Out of the 5 annual targets for FY 2005 only one aggressive milestone was missed. Overall, since the Program's inception in 2002 the Program has had 27 targets and only missed 3 of those targets. The 3 missed targets have not jeopardized the Program's achievements towards its long term goals. In addition, based on the lessons learned from an HTS missed target in 2002, a program risk mitigation activity ("Readiness Reviews") was begun to identify potential failure areas and deal with problems prior to proceeding with the next stage of projects.

Evidence: 1. FY 2007 Congressional Budget Request 2. Individual Project Peer Reviews (sensitive information, internal use only): i) HTS Peer Review, ii) Energy Storage Peer Review, iii) Distribution Transformation Peer Review, iv) Transmission Reliability Peer Review, 2005, (for 2004 report go to: http://www.oe.energy.gov/transre4.cfm) v) Distributed Energy Peer Reviewers (http://www.energetics.com/depeerreview05/agenda.html) 3. Project Performance Reports 4. Explanation of HTS Missed Milestone 5. Storage Milestone Explanation

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: As discussed in question 3.4, OE's policy and practice is to utilize competitive contract awards whenever possible for both programmatic and program support activities that are discretionary (i.e. non-Congressionally directed) in order to achieve efficiencies and cost effectiveness, including an overhead efficiency of less than 12% for R&D (in accord with DOE's definition for the R&D overhead efficiency measure). The Program has achieved and maintained target levels for FY05 and FY06 below 8%.

Evidence: 1. OE Efficiency Measure

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Program was reviewed and compared to other research organizations including the CEC and 3 other federal R&D programs. The OE R&D Program was compared in 6 aspects of its portfolio and it compared favorably. These 6 aspects included: 1) Technical Excellence, 2) Innovation, 3) Technical Feasibility, 4) Program Risks, 5) Uniqueness, 6) Rewards vs. Risks.

Evidence: 1. OE Portfolio Management External Benchmarking, March 2006, Navigant Consulting 2. Portfolio Assessment??FY04 Results OETD & PIER Programs, October 2004, Navigant Consulting

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Comprehensive Peer Reviews by independent reviewers have been conducted on a regular basis. They rate the effectiveness of the Program and are the official program evaluations. The independent peer reviewers are asked to evaluate not only the individual projects, but also the management and alignment of the program with its intended mission and performance measures. Results from all peer reviews have found that the electric system R&D program is of high quality, sufficient scope and effective. Although some individual projects/contracts have been rated deficient in some aspect, the program as a whole has demonstrated no managerial deficiencies. The HTS subprogram, for instance, has been acknowledged for its leadership role in revolutionizing the electricity delivery system. In addition, for the past three years, there have been 7 R&D100 awards for OE research and a 2005 Utility Automation & Engineering T&D Project of the Year Award in the category of transmission and distribution engineering. These independent evaluations indicate that the program is effective and achieving results.

Evidence: 1. Individual Project Peer Reviews (sensitive information, internal use only): i) HTS Peer Review, ii) Energy Storage Peer Review, iii) Distribution Transformation Peer Review, iv) Transmission Reliability Peer Review, 2005, (for 2004 report go to: http://www.oe.energy.gov/transre4.cfm) v) Distributed Energy Peer Reviewers (http://www.energetics.com/depeerreview05/agenda.html) 2. EERE Peer Review Guide, August 2004 3. Portfolio Assessment??FY04 Results OETD & PIER Programs, October 2004, Navigant Consulting