Detailed Information on the
US Geological Survey - Energy Resource Assessments Assessment

Working with external partners on geothermal and gas hydrates performance and goals found in the Energy Resources 5-year plan.

Implement new functionalities on the Energy Resources web site to help users to find data and information more easily.

Refine performance measures drafted during the PART process and develop a five year program plan that is consistent with these measures.

Work to continue to make reports and data more accessible and user-friendly.

Monitoring actual performance against performance measures and goals in the new 5-year plan.

Deploying and evaluating redesigned Energy resources web site to ensure it meets user needs.

Validate program performance alignment with measures and goals in the 5-year plan, with emphasis on Energy Resources Program goals 3 (hyrdates) and 7 (partnerships).

Implementation of redesigned Energy resources web site to ensure it meets user needs

Validate program performance alignment with measures and goals in the new 5-year plan, with emphasis on ERP goals 4 (geothermal), and 7 (partnerships)

Monitor and expand data delivery from the redesigned Energy Resources website.

Measure: % of studies validated through appropriate peer review

Explanation:A study is a systematic analysis, a significant body of knowledge that improves the understanding of natural process, resources, and ecosystems typically resulting from several years of research assessment activity. Peer review is a criterion for OMB Results Not Demonstrated investment. This measure acknowledges this requirement. The numerator and denominator are studies and peer reviewed studies. This measure acknowledges adherence by all USGS scientists to fundamental science practices and the OMB quality criterion for R&D investments. The population it covers is the systematic analyses and investigations -- output/products -- of the programs' research .

Measure: % of targeted science products that are used by partners or customers for land or resource management decision making

Explanation:This measure is tracked by survey of customers and partners.

Measure: Number of targeted basins/areas with energy resource assessments available to support management decisions

Explanation:Targeted basins/areas are those basins/areas with energy resources that have been determined to be important for resource estimation. Selection criteria are usually based upon resource volume or presence of resource on Federal land, but may also be legislatively determined, such as those identified in the Energy Policy Act of 2005. Basins/areas in FY2008 delivered: Oil and gas assessment of the Permian Basin (TX and NM), oil and gas resources of the Bakken Formation of the Williston Basin (WY and MT), biogenic gas in northcentral MT, the Circum-Arctic Petroleum Resource Appraisal, and the national (U.S.) geothermal resource assessment. The research needed to conduct an assessment varies from basin/area to basin/area, as each area is geologically unique in terms of setting, history, and type of energy resource contained. Each area assessed takes several years to accomplish.

Measure: Average cost of a systematic analysis or investigation

Explanation:Average cost per analysis allows comparisons among different projects to determine how efficiencies can be achieved. However,each assessment is unique in terms of what data are needed to be obtained (sometimes purchased), what type of geology is present, what new samples need to be analyzed, and so on. While our understanding of energy resources and our expertise in conducting geologic analysis carry over from area to area, each basin or area must be assessed afresh with its attendant data. The cost of an assessment varies considerably depending on the detail of the study, the geographic locality and size of area of the study, the length of the study, amount of information already known, whether data must be purchased or collected, and so on. Further, once an assessment is done, it must be redone in a few years, as the economics, technology, and geologic understanding of resources change.

Is the program purpose clear?

Explanation: The Mission of the Energy Resources Program (ERP), as stated in the program's 5 year plan and other planning documents, is to assess the energy resource potential of the Nation and the World (exclusive of U.S. Federal offshore waters) and the environmental and human health impacts of energy production and use in order to plan for a secure energy future and allow for the strategic use and evaluation of resources.

Evidence: Legislative mandates (1.1A). The ERP mission is consistent with: (1) the mission and goals of the DOI Strategic Draft Plan (2003-2008) (1.1B) under Mission Area "Resource Use" - Manage or influence resources to enhance public benefit, promote responsible use, and ensure optimal value - Energy; DOI Strategic Goal "Manage natural resources to promote responsible use and sustain a dynamic economy;" (2) the USGS Strategic Plan (2000-2005) Mission Goal to "Provide science for a changing world in response to present and anticipated needs to expand our understanding of environment and natural resource issues on regional, national, and global scales and enchance predictive/forecast modeling capabilities;" (3) the Geology Science Strategy (2000-2010) Goal 3 -- Advance the understanding of the Nation's energy and mineral resources in a global geologic, economic, and environmental context (1.1C). The NRC review of the ERP (1999) specifically states that the role of the ERP is clearly defined, fulfills a mission essential to the federal government, and is unique from that of other federal agencies (1.1D).

Does the program address a specific and existing problem, interest, or need?

Explanation: The ERP is responsible for assessing national and international energy resources and conducting research in order to perform those assessments. A sound, scientific knowledge base is needed to assess available resources and the impact of using those resources, and to put such information into a context as to allow decision makers to understand and weigh the costs, risks, and benefits of energy usage. The ERP addresses these challenges by generating and providing objective, science-based energy information essential for shaping policies regarding domestic and foreign energy resources, making sound decisions regarding Federal land use, and maintaining a healthy domestic energy industry.

Evidence: Energy is one of the most important components of the world's economy. The U.S. is 85% dependent upon fossil fuels. Net energy imports have risen in the last 2 decades and total energy consumption is expected to increase more rapidly than domestic production, requiring increased net energy imports (EIA, 2003 - 1.2A). Adequate and reliable supplies of affordable energy, obtained in environmentally sustainable ways, are essential to economic prosperity, environmental and human health, and political stability. The NRC review (1999) (1.2B) stated that "the products of the ERP are important to the economic, environmental, and security future of the U.S." EIA states that "The USGS petroleum assessments provide an important foundation for geologic, economic, geopolitical, and environmental studies. With many of the world's economies intrinsically linked to energy resource availability, such studies provide essential long-term strategic guidance" (1.2C). The National Energy Policy (1.2D) cites ERP NPRA and ANWR 1002 assessments as the authoritative estimates of undiscovered oil and gas resources in Alaska.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: The ERP role is clearly defined and unique from other Federal, state, local, or private entities. The ERP focuses research to define the geologic factors that control the abundance, distribution, quality, and location of energy resources. ERP research focuses on defining technically recoverable undiscovered oil and gas resources, coal resources and reserves, coalbed methane, framework and process studies for all these commodities as well as gas hydrates, and the environmental and human health factors associated with the production and use of energy resources.

Evidence: Other Federal agencies that work on energy-related issues (MMS, BLM, USFS, DOE, EIA) do so in mutually exclusive areas (1.3A). The NRC review clearly stated ERP's unique role in this regard.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: ERP is designed to conduct research and scientific assessments on energy resources. ERP employs an expert federal workforce with extensive experience in energy research, assessment, geochemical, and related expertise, such as IT. Based upon feedback ERP actively seeks, projects' purposes, methodologies, and scopes have evolved over time. ERP partners with others (federal agencies, states, academia, industry consortia), and by leveraging funding, expertise, and facilities, ERP maximizes the impact of science accomplished, lessons learned, and products produced. Laboratories are operated at regional USGS centers where costs are shared with other programs.

Evidence: To gauge ERP's effectiveness and evolve its energy research, stakeholder and partner feedback is actively sought by many venues, including: (1) NRC reviews (see 2.8 for specific steps taken by ERP from NRC review feedback), (2) customer surveys (1.4A), (3) interacting with customers at scientific and technical meetings, (4) calling and e-mailing customers, (5) ERP membership on interagency steering committees (1.4B). ERP scientists develop state-of-the-art methodologies and techniques for energy assessments and are recognized leaders in this field. ERP seeks outside validation of its methodologies, assessments, and studies to ensure that no major flaws are present.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: ERP's mission concentrates on providing original, geologically based, non-biased energy information to policy makers, land and resource managers, other federal agencies, foreign governments, nongovernmental groups, industry, academia, other scientists, and decision makers. Program funding is directed at achieving program goals, namely understanding and assessing the fossil-energy resources of the Nation and the World and the environmental impact of energy resource production and use.

Evidence: ERP's funds go to scientific projects dedicated to the program's goals and mission. Funding procedures follow USGS and GD guidelines which are outlined in section 3. However, energy resource information is the "real" ERP resource that reaches intended beneficiaries. ERP research plays a role in shaping U.S. and international policy, as indicated in press release in (1.5A). Because ERP's purpose is to provide energy information, ERP systematically and proactively seeks feedback from intended beneficiaries, in order to determine that products are useful, data are timely, methodologies are sound. To seek meaningful feedback, ERP employs a number of mechanisms (also outlined in 1.4): (1) Customer Surveys (1.5B); (2) Customer Listening Sessions (1.5C); (3) web statistics (1.5D); (4) scientific and technical stakeholder meetings; (5) calling and e-mailing customers/partners; (6) participating in interagency steering committees (1.4B). The ultimate beneficiary of ERP's work is the public. Results from ERP studies are in GEODE, are found on ERP web sites, on CD's available to all, and are presented at scientific and academic forums.

Does the program effectively articulate potential public benefits?

Explanation:  

Evidence:  

If an industry-related problem, can the program explain how the market fails to motivate private investment?

Explanation:  

Evidence:  

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program did not have long term measures that focus on outcomes. The measures largely focus on outputs and process (customer surveys). Goals in the 5 year plan were not specific enough to evaluate performance. New measures were developed in the PART process.

Evidence: Outcome goals of the draft DOI Strategic Plan (Resource Use - Energy) include "improve information base, resource management and technical assistance." The current ERP 5-year plan lists 4 major objectives that encompass the work conducted by ERP in order to fulfill its mission. These objectives are: (1) To understand and assess the fossil-energy resources of the Nation and the World; (2) To understand the geologic framework and processes of energy resources; (3) To understand and evaluate the environmental impact of energy resource production and use; (4) To deliver energy resource information to land and resource managers, energy policy makers, other scientists, academia, private industry, environmental groups, and other non-governmental entities. All ERP-funded projects support one or more of these goals.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Long term targets and timeframes for ERP are listed as 5-year objectives in the program's 5-year plan. It is difficult to detemine whether targets are ambitious for the following reasons: program goals and the narrative for 5 year plans are too broad to be considered measures, they do not include time frames or specific products. Annual project work plans contain more detail and time frames, but are not clearly linked to achieving goals in the 5 year plan.

Evidence: ERP 5 year plans, GPRA reports.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: Annual performance measures are representative of the overall, broader program mission and goals and serve to illustrate progress on assessing the Nation's and World's energy resources and the impact of their development. GPRA goals address annual performance by reporting, on a quarterly basis the following: number of systematic analyses and investigations delivered to customers, number of long term data collections maintained, number of stakeholder meetings, and number of data delivery systems maintained. All these measures directly relate to the long term goals listed in 2.1 and the ERP 5 year plan. All annual goals support the long term goals outlined on the measures page. Illustrations showing the connection between ERP long and short term goals, GPRA, the GD Science Strategy, USGS Strategic Plan, DOI Strategic Plan, and the President' Business Reference model are found in (2.1A). Each annual measure achieved provides evidence of progress towards long-term goals, but there is not sufficient information to detemrine adequate progress.

Evidence: Annual performance measures demonstrate progress toward ERP's long-term goals and are found in GPRA measures and project plans.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: Baselines and targets for ERP projects are listed in the GD Annual Science Plans, annual project work plans and proposals, and in annual federal budget justifications. ERP-funded projects develop new project proposals every year that are consistent with ERP program priorities and goals, to report progress, and make necessary changes to project direction. These project proposals are reviewed annually by a Program Council composed of senior energy scientists, both internal and external to ERP. The Program Council makes recommendations on project progress and ensures that the targets are appropriate, ambitious, and obtainable. Annual measures are documented in tasks for each project and reviewed at the end of the year before additional funds are given out. ERP-funded teams are also reviewed annually by an internal USGS review team and periodically by an external to USGS review team.

Evidence: Baselines and targets include project funding projections, annual project proposals/plans, and targets are established through budget initiatives and annual budget planning. One project proposal, for Alaskan Petroleum Studies for FY2001, FY2002, and FY2003 is included in (2.4A). All project proposals contain objectives, strategy, impacts, products, collaborators, work plans, outreach, publications proposed and delivered, and accomplishments. The Alaskan Petroleum Studies project assesses the hydrocarbon resources of Alaska - an ambitious undertaking. Alaska contains some of the largest hydrocarbon accumulations in the U.S. and is one of the most hotly debated areas in the world. To meet all short and long term targets, this project outlined its annual and long term priorities, worked consistently toward understanding the framework and processes of hydrocarbon occurrrence in Alaska, processing and interpreting seismic information, etc. in order to assess the resources in Alaska in a timely fashion. See also gas hydrate example of project annual goals related to ERP long term goals in 2.3.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ERP does not fund any work that is not directly linked to and in support of ERP research, annual, and long-term goals. ERP has no grantees, but does have a Cooperative Agreement program with a number of State Geological Surveys. All partnerships and agreements are covered by some formal document (MOU, letter of intent, statements of work, etc.) outlining duties, expectations, and products. All contracts must have a statement of need and be directly tied to an ERP-funded project in order to be approved. As program goals are broad, it is difficult to evaluate the performance impact of partnerships.

Evidence: Where appropriate, ERP forms partnerships to work with others with specific needs, data, knowledge, facilities, where ERP has been asked to help or where there is an ERP need to meet its annual and long term goals. Some partnerships are collaborative and expertise only is shared; others are cost-shared, in terms of facilities, data, or funding (e.g. 2.5A is an agreement between ERP and Geological Survey of Canada for gas hydrate research). Other agreements are in-kind - CRADA's developed with industry and academia to explore for and research coalbed methane (2.5B). Both of these kinds of agreements contribute to the short term goals of understanding the nature, occurrence, and distribution of the resource (gas hydrates or coalbed methane) to fulfill the long term goal of assessing the technically recoverable resource. Other agreements are funded by partners (2.5C - Interagency Agreement with BLM for coalbed methane) where ERP possesses an expertise needed by another agency, thereby providing the scientific information necessary for BLM to produce their Environmental Impact Statement.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Comprehensive, independent program reviews are conducted by the NRC on a periodic basis (approximately every 5 years). ERP uses these reviews to help improve ERP performance, efficiency, and effectiveness. In addition, all projects are reviewed annually by a Program Council composed of senior energy scientists internal and external to the program to ensure progress on project and program goals and to adjust work as needed to meet long term goals, customer needs, emerging energy needs, and evaluate effectiveness and relevance of project work. Furthermore, when major program elements are developed, such as methodologies for oil and gas assessment, ERP seeks outside review and validation. ERP seeks customer input and feedback on a regular basis. All ERP projects also seek feedback from customers, partners, and stakeholders to determine relevance and effectiveness and project development. ERP-funded teams are reviewed annually by an internal USGS review panel and periodically by a review panel external to USGS.

Evidence: The NRC reviews ERP periodically (approximately every 5 years). NRC recommendations are incorporated into ERP practices (details are found in 2.8). As noted in 1.4, ERP sought scientific and technical validation of its oil and gas assessment methodology. Also explained in 1.4 is the ERP-NPC working relationship to develop economic models for unconventional oil and gas. ERP members belong to a number of interagency steering committees that meet regularly, not only to discuss areas of mutual interest, but to obtain feedback of goals, methodologies, products, etc. A few if these interagency committees are listed in (2.6A). ERP members belong to international standardization bodies. ERP assessment projects actively seek input from state organizations and the private sector when developing the geologic models upon which the assessments are based. ERP outside validation comes in many forms - one example is that the ERP World Energy project was 1 of 3 finalists at the Institute of Petroleum for the International Platinum Award, for great international impact and innovation.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Program budgets are not clearly tied to annual and long term performance goals. The items listed in the GPRA table are not clearly tied to descriptions of actual acitvities within the text of the budget materials. Further sufficient, measurable long term perforamnce measures did not exist to determine whether the budget was sufficiently tied to performance.

Evidence: ERP 5 year plan, Project Work Plans, Congressional Justifications.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The NRC review of the program identified a number of areas for program improvement, which have been implemented. The program 5-year plan has been aligned with the USGS and GD Strategic Plan and will be aligned with the DOI Strategic plan. Continuous strategic planning in the ERP anticipates changing policy environments and new developments in science and technology and identifies evolving needs for scientific and technical expertise. Continuous customer and stakeholder feedback also helps to shape ERP direction.

Evidence: ERP (1999) acted upon the recommendations of the NRC (1999) review: (1) NRC: maintain a strong research and knowledge base - ERP: after several years of focus on resource assessment, ERP separated framework and process studies from resource assessments, which aids in developing state-of-the-art assessment methodologies; (2) NRC: improve communication between the oil, gas, and coal sub-programs - ERP: ERP combined projects on a regional basis (Gulf Coast, Appalachian Basin, and Alaska) and these projects study all commodities in the region; (3) NRC: broaden ERP's portfolio to include all geologically based energy resources - ERP: ERP substantially developed coalbed methane and gas hydrate projects and developed a project entitled "Alternative Energy Resources of the Future" to provide periodic updates on the status of other commodities, such as heavy oil and oil shale that are predicted to remain a minor part of the energy mix in the next few decades.

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: The program does not assess and compare potential benefits to other efforts that have similar goals. ERP has a unique role and mission. but has a similar goal of better understanding energy resources as programs in state geological surveys and programs at DOE such as the Geothermal Energy program. There are demonstrable benefits to other programs, efforts, and organizations from the program. ERP does evaluate the efforts within the program to attain the best products and advances toward the short and long term goals possible. Though a formal cost/benefit has not been performed for ERP, cost benefit studies of other programs with open access to information policies suggest making information publicly available increases benefits to society. The NRC recognized benefits of ERP when it stated, "A significant duplication of effort would be the result if agencies were to develop internally the information provided to them by the ERP".

Evidence: ERP focuses its efforts on geographic areas, commodities, or studies that will further its short term and long term goals the most - usually in areas where there is relatively little known about the commodity. ERP compares potential benefits of projects within the program in order to balance basic and applied resources, while maximizing outputs and outcomes. ERP focuses on those resources with the greatest potential for meeting the nation's energy needs. ERP focuses on traditional resources of oil, gas, and coal, as well as frontier resources such as gas hydrates and coalbed methane, and only to a very small extent on resources such as oil shale and heavy oil. The knowledge gained from studying gas hydrates will substantially benefit the nation, especially as start up time for producing such new commodities is usually 5-10 years. Research is needed now to be prepared for when the technology and economics are conducive to development. ERP also focuses its efforts in frontier areas, such as Alaska, where little information exists, in order to provide the basic scientific information needed to make sound policy decisions. NRC review of ERP (pg 2)

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The ERP uses a rigorous prioritization process as described above in sections 2.1 through 2.7, 2.RD1, and in section 3. The ERP takes into account and balances long range goals with current affairs, legislative mandates, requests from DOI and other DOI bureaus, stakeholder needs and input. Annual project plans are reviewed by the ERP Program Council to help determine prioritization, direction, mid-project course adjustment. Within project prioritization is critical to success as well, because many ERP projects, assessment projects in particular, are long term and ambitious and have many tasks to accomplish. The 5 year plan has listed priorities, but priorities are not clearly communicated through long term performance measures.

Evidence: ERP uses a rigorous prioritization process, employing long term goals and annual project planning and review. The long term and annual measures, goals, targets, customer feedback, partner input, etc. all described in 2.1-2.7 go toward prioritizing funding decisions. Each project submits a proposal every year. This allows the ERP Program Council to annually review each project and allows for a recalibration of the program every year. The ERP Program Council, with rotating membership of energy experts from ERP, other programs, and outside organizations, reviews projects to identify new ideas and partnerships, bring new expertise and perspective to project decisions, and to help ERP identify stakeholder needs. The prioritization process is housed under the broad goals and objectives of ERP, with annual adjustments determining what is of priority. Priorities in any given year include U.S. and global oil, gas, and coal assessment activities, research in support of these assessments and other agencies' activities, and research where ERP contains significant expertise and has much invested such as gas hydrates, coal quality, and coalbed methane.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: DOI,USGS, and its Programs regularly collect performance information through customer and partner reviews and surveys. Feedback is incorporated into program plans and specific actions are taken in response.The DOI and Bureau Strategic Plans include partner and customer reviewed long term goals, annual performance measures, and GPRA measures. Progress on GPRA are verified quarterly and reported and updated annually. The USGS Director convenes annual listening sessions, recording needs of partners and informing them of response. The NAS/NRC reviews long term goals and program performance, utilizing blue ribbon panels of scientists and stakeholders. All projects are required to record detailed workplans, progress and products, and budgets by object class in the Bureau wide system BASIS+. Projects workplans and grant proposals are reviewed annually by Programs using advisory panels. Written feedback on performance is provided to project chiefs who must correct deficiencies or suffer budgetary penalties for non performance.

Evidence: Documents: Bureau Strategic Plan showing long term goals, measures, and annual GPRA targets (pp 9-15). GPRA update memo for FY-02, GPRA Reports for 03 and example of quarterly verification. Directors 03 Listening Session Report showing recommendations and actions taken. USGS Planning Model process showing performance requirements in program five-year plans (p.9) and collecting performance information in BASIS+ system (p.12-13). Overview diagram of planning process, Geology Strategic Plan, Geology Science Policy, Geology Annual Science Plan and example project of National Seismic Hazards Map. Energy Resources Program: ERP collects timely and credible performance information from a variety of sources: NRC reviews (see section 2.8 for detailed explaination of how ERP incorporated NRC recommendations); interagency steering committees that meet regularly, such as the EPCA committee (composed of USGS-ERP, BLM, USFS, DOE, EIA), which meets at least 4 times a year; annual Program Councils to review all project plans and progress; and whenever a major methodology is developed or product released.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: USGS holds senior management and program partners accountable for performance through performance evaluation, management process controls, and performance guidance provided in agreements, contracts, and grants. Measures for GPRA, financial management, and the Presidents Management Agenda are in all USGS SES performance agreements. Regional Executives and Program Coordinators are accountable for achieving performance as part of the USGS Planning Model. Grant programs have specific performance guidance and include rigorous review panels and budgetary penalties for non performance. Cooperative agreements with states and universities include specific requirements, products, and time schedules with payment penalties for non performance. Contracts for services are competed and contain specific quality and performance requirements and time schedules for services.

Evidence: Documents: SES Performance Plan Guidance and Trujillo Memo, USGS Planning Model responsibilities list (p.4-7). Contract and agency agreement requirements from the USGS Policy Manual. Energy Resources Program: All ERP partners are held accountable for cost, schedule, and performance results. One example is found in (3.2A), Assistance Award for "Assistance in the Development of Coal Resource Assessment Classification System and Evaluation of Coal Quality Data in the USGS Database." The contractual document contains a Scope, Statement of Work, Deliverable and Schedule for such, Background information on why such an Award is necessary, and contains the sentence "Final acceptance will be made upon completion of the work as required and data and reports delivered to the U.S. Geological Survey."

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The USGS has an established budget, allocation, and spending process that includes annual planning, quarterly and monthly reviews, and review of any funds allocation change over 25K. It has implemented management controls and measures to ensure dollars are allocated and obligated in a timely manner and spent for intended purposes. Budget planning to object class is done in the BASIS+ system, which ties budget to intended use. Allocation tables are constructed from BASIS+ and FFS is used to provide monthly and quarterly spending information by object class, to review obligation and debt, and take corrective action. Projects and their budgets are reviewed monthly by line managers and annually by Programs. Changes of over 25K are reviewed by both regional line managers and Programs as they occur. The Bureau conducts quarterly review of status of funds against performance measures. A certified Contracting Officer's Representative annually reviews and verifies contract funds are obligated and spent for intended purposes.

Evidence: Documents: USGS Budgeting and Finance diagram. FY02 Geology Annual Science Plan showing project science and funding targets. FY02 Allocation Process Memo showing appropriation actions and requirements. FY02 Program and admin office allocation tables to cost centers, projects, and accounts. These numbers are consistent with budget numbers in FY-02 Geology Annual Science Plan. FY02 National Seismic Hazard Map project/budget and FFS reports showing FY02 cost center spending on National Seismic Hazards Maps and dollars spent for intended use at project level. Spending progress by object class for all USGS for '02 2nd and 3rd quarters. Summary of Program quarterly obligations for FY02 showing consistent spending of appropriations for intended program. Final spending report for all FY02 Programs. Instructional Memos APS-2003-11-13 showing monthly management control requirements for accounts receivable, unbilled balances, and obligations-accruals-changes to allocations > 25K. Description of cost centers use of FFS monthly reports to inform account holders of spending progress and funds available.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The Bureau is engaged in competitive sourcing for Visual Information Services, Building and Ground Maintenance, and Warehousing. These competitions will improve cost and timeliness of program publications and exhibits and the warehousing we use for major program assets. Geology mission critical information systems have submitted Capitol Asset Plans (Exhibit 300) to DOI and are in the certification and accreditation process. Geology programs are gaining effeciencies in timeliness and cost by serving digital data and analysis tools through common portals. In 2003, all Geology programs are developing Activity Based Costing for 2004 implementation. ABC will allow for comparisons of overhead costs across programs ro research and assessments. Scientists are required to submit annual project work plans and budgets for review of progress, performance, and cost.

Evidence: Documents: April 2002 Memo from USGS Director announcing competitive sourcing, June 2003 update on competitive sourcing. DOI Capitol Asset Guidance. Examples of digital data initiatives and portals. Geology Science Planning Policy. Energy Resources Program: ERP IT improvements are numerous and some details are found in section 4.3. Another example is that ERP's geophysical processing project initiated a contract to provide a fast acquisition path for purchasing reflection seismic data from commercial vendors; ERP and other programs use this contract to acquire such data at a reduced cost and a timely manner; cost benefits of acquiring the data without the contract as compared to acquisition with the contract show that ERP has saved $800,000 in the past 3 years - funds that can be used for other program purposes. The ERP Central Energy Team recently registered its services and data with a geospatial resource web portal called Geography Network, which caused usage to significantly increase after registration.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Geology Programs collaborate with federal, state, and local governments, industry, and academia towards the achievement of complimentary goals. Major partners are identified in the Geology Strategic Plan and in Program Five-Year Plans and include, but are not limited to, DOD, NSF, DOE, EPA, BLM, EIA, and DOI, State Geological Surveys, state and local resource agencies, and major consortia of academic, governmental, and industry groups. In general, USGS provides the broad scientific framework that provides context and support for partners to conduct work on a more specific or local basis. USGS establishes roles and responsibilities with partners through cooperative agreements, Memoranda of Understanding, or Cooperative Research and Development Agreements (CRADA).

Evidence: Effective collaboration between ERP and others is evidenced by many working agreements ERP has with others. Listed in section 1.3 are some of these partnerships - gas hydrate research with DOE, MMS, BLM; coalbed methane work with BLM and industry consortia; BLM collaboration with EPCA; and many others, including State surveys. In all of these relationships, ERP plays a distinct and complementary role. Example: in the EPCA work, ERP provides estimates of undiscovered oil and gas resources underlying Federal lands and BLM provides the surface restrictions and impediments to development. ERP cooperative working relationships benefit both parties, and ERP scientists gain access to data, knowledge, and expertise as well as funding. Sometimes these relationships have other tangible benefits: members of the ANWR consortium requested ERP reprocessed reflection seismic data from ANWR. In return for this reprocessed ERP data, ERP acquired a corresponding number of miles of company owned seismic data at no charge. To acquire these data commercially would have required an expenditure of more than $1.5 million.

Does the program use strong financial management practices?

Explanation: The 2002 Audit findings of the Inspector Generals Office conducted by KPMG contained a "no opinion" result and cited 8 reportable conditions in their report dated January 24, 2003. Due the extent of financial management problems cited in the audit and the inability of the auditor to render an option, it is difficult to separate the program from cfinancial magnement problems. USGS submitted a Corrective Action Plan that has been accepted by the Inspector Generals office. In his cover memo, the Asst. Inspectors General for Audits stated: "Based on the response and corrective action plan, all the recommendations are considered resolved but not implemented." Monthly meetings and reports on progress are being provided to DOI and thus far, many tasks are completed and all others are in progress. In the USGS matrix organization, line management and administration is responsible for financial, facilities, and personnel management. USGS Program Coordinators are responsible for scientific planning and coordination, budget formulation, and establishing and reviewing performance.

Evidence: Documents: April 11 Auditors Report 2003, Corrective Action Plan, and cover memo from Asst. Inspector General for Audits Roger LaRoche.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The USGS is taking the necessary steps to resolve management deficiencies. The USGS has aggressively addressed IT control weaknesses. Management control performance measures have been incorporated into all SES Performance Evaluations. An expert team has been formed and operating for the last 6 months to address audit issues and ensure completion of the Audit Corrective Action Plan. Extensive training is underway to address reported conditions and strengthen management practices. Administrative Instructional Memoranda outline in detail all financial processes and requirements. All Geology Programs use an annual review process and the BASIS+ system to review all program work and correct deficiencies. This is described in detail in 3.4 and 3.RD1. The NRC and FACA advisory panels conduct periodic reviews that make recommendations regarding program management, performance, and scientific direction.

Evidence: Documents: Corrective Action Plan Progress Report submitted to DOI showing progress or completion of all actions. Hord Tipton Memo providing improved results of March and April testing of DOI WAN's. Instructional Memoranda from 3.3. ERP is proactive in taking corrective measures when needed: (1) In the past, ERP released assessments periodically (~every 5 years). After the 1995 national assessment, with input from stakeholders, Program Council review, and critically looking at funding, FTE, proposed targets, partner needs, ERP made a systematic change in the process to "rolling assessments." Now, when an assessment is done, it is released rather than waiting for other regions to be finished, thus increasing ERP's timeliness and allows for concentration of limited resources. (2) From the post-appraisal (e.g. customer feedback) of the National Coal Assessment, ERP determined the next phase of assessment would be a methodological revision. Two existing coal projects were combined into 1 to streamline assessment functions. The letter outlining this decision and the prospectus before the combination and after are found in (3.7A).

Does the program allocate funds through a competitive, merit-based process, or, if not, does it justify funding methods and document how quality is maintained?

Explanation: Since 1996, Geology Programs have been a leader in conducting a division-wide competitive project proposal process using a prototype of the BASIS+ system now in use across the Bureau. Geology issues an annual call for project proposals called the Geology Annual Science Plan (also known as the Geology Prospectus) which contains scientific and funding guidance for all projects. The annual plan uses the Geology Science Strategy and Program five year plans for its organizing framework. Scientists are required to submit annual project proposals into the BASIS+ system for program review. The system is used to examine strengths and weaknesses in staff, scientific methodology, progress on goals, budgetary structure, use of funds and capital investments, and formulate final allocations. Reviews are conducted by scientific peers and include external scientific or stakeholder review. Earmarked funds are not excluded from review.

Evidence: Documents: Overview diagram of Geology Planning Process demonstrating management and review process. See also answers to 3.1 and 3.3 on planning and allocation processes.

Does competition encourage the participation of new/first-time performers through a fair and open application process?

Explanation:  

Evidence:  

Does the program adequately define appropriate termination points and other decision points?

Explanation:  

Evidence:  

If the program includes technology development or construction or operation of a facility, does the program clearly define deliverables and required capability/performance characteristics and appropriate, credible cost and schedule goals?

Explanation:  

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: ERP completes projects that are related to long term goals in energy resource assessment, improved development of assessment methodologies, framework geologic studies, and research studies of environmental and human health impact of energy production and use. The goals in the five year plan are too broad, and do not have baselines or targets to determine if adequate progress has been achieved. While activity and timeline information is collected at the project level it is not clearly linked to long term goals.

Evidence: Examples of ERP contributions to long term performance goals: (1) The World Petroleum Assessment is the world reference standard for international policy development - it is the reference case for IEA's World Energy Outlook (2000,2001, 2002), EIA's International Energy Outlook (2000, 2001, 2002), and the benchmark reference case used by climate modelers at Stanford, MIT, and PEW Center; (2) The results of ERP's assessments of oil and gas resources in Alaska are considered the most objective available and used by the Administration and Congress in formulating energy policy, increased industry interest in NPRA before the lease sales, and used by most groups debating the pros and cons of development in this area (4.1B and 4.1C); (3) Because of ERP's nonadvocacy role, the Northern Cheyenne Tribe requested ERP assistance to evaluate the potential of coalbed methane resources on the Reservation (4.1D), information which will be used to set tribal policy.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Baseline and targets for ERP-funded projects are listed in annual science plans (the Geology prospectus), annual project work proposals and plans, and in documentation of base and initiative budgets. Cooperative and other agreements with other federal agencies, states, academia, etc. all have outlined periodic, usually annual, targets in the written document. See also sections 2.4 and 2.5.

Evidence: Annual priorities are established in the context of long-term goals (ERP 5-year plan, GD and USGS science strategies, DOI strategic plan) and performance is checked by management and Program Council review, approval and funding of both internal USGS projects and external partners. Project funding is adjusted annually based on performance and programmatic priorities. Annual targets are reported every quarter in GPRA. All contractual and working agreements have annual performance goals. See e.g. Assistance Agreement (4.2A) between BLM and ERP which outlines specific deliverables for the fiscal year for gas hydrate work. The EPCA inventory had a congressionally mandated deadline. To complete the work required in the short time available, very tight performance goals were necessary; targets were achieved and the interagency report was released on time. ERP assessments conducted in Alaska (see also 2.4) were done on time, in order to meet a variety of goals, such as conducting workshops open to all to create interest in NPRA lease sales in the time frame required by BLM. All annual goals work toward the long term goals.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: ERP has adopted a number of new technologies, methodologies, ways of conducting science that have increased efficiencies and cost effectiveness. ERP's effective use of IT and GIS have increased its efficiency tremendously (see also 3.4). While savings have occured, there is no regularly collected data which facilitate cost effectiveness determinations over years, or permit comparisons across programs.

Evidence: ERP maps are now digital and most are GIS-based, significantly increasing their usability to a variety of users, not just traditional ones. ERP products are available over the web, reaching a much wider audience than in the past. Digital products have reduced costs for archiving and distributing products. The NERSL project has modernized its storage and near online retrieval systems, switching from CD-ROM to DVD capabilities increasing the amount of storage space and speeding retrievals. The geophysical processing project has evolved tremendously and provides seismic data acquisition, processing, and interpretation support to ERP and others; having an in-house service provides faster service and saves ERP a large amount of money (approx $2 million over the last 5 yrs) which can be used for other scientific endeavors. In the past, ERP produced national assessments periodically. Now, ERP produces "rolling assessments," so that products are delivered in a more timely and efficient manner. Partnering has increased significantly to take advantage of expertise where it exists and fill short term skill needs.

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: Though there is not specific performance data to compare with other efforts, NRC reports and program partners suggest the program However, many other programs have complementary missions. The ERP works with these other groups or they use data produced by ERP projects for their work.

Evidence: The NRC review of ERP specifically stated that ERP's role is clearly defined and unique (4.4A). Many agencies (listed in 1.3) and the private sector use ERP information: (1) oil and gas assessments feed directly into EIA forecasts; (2) See e-mails (4.4B) describing use of results from the organic geochemistry lab and other ERP projects; (3) Environmental groups use ERP assessments as the basis of their studies (4.4C); (4) The financial community requests "reality checks" on loan requests that use oil or gas as loan collateral and the IRS consults ERP concerning tax policy and reserve growth; (5) The NPC (a DOE FACA), charged with studying natural gas supplies is "using the USGS assessments as the basis" for their study (4.4D); (6) The NPC also requested ERP expertise in gas economic analysis (4.4E); (7) BLM requested ERP assistance in carrying out environmental responsibilities in land planning (4.4F). An important ERP role is to archive information, maps, data that would otherwise be destroyed or disintegrate, such as the NPRA data set rescued from NOAA (originally a billion dollar+ investment) and archived by ERP (4.4G).

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Independent review of the ERP by various groups have found the program to be effective and achieving results. These groups include the NRC, American Association of Petroleum Geology Core Committee, National Petroleum Council, and others as needed, such as the independent Geode review. Also testament to ERP's effectiveness is the vast amount of feedback ERP receives in the form of e-mails, letters, and verbal information indicating the breadth and scope of use of ERP products.

Evidence: The NRC review of ERP states: "the mission of ERP - to provide up-to-date and impartial assessments of geologically based energy resources of the nation and the world - is fully appropriate for a federal earth-science agency. The information and data are essential to the management of federal lands, to the understanding of the environmental impacts of the extraction and use of energy resources, and to the planning of national energy policy." Other examples: (1) AAPG reviewed/endorsed ERP assessment methodology (4.5A); (2) Other organizations use (OPEC) or adopt (Australian gov't) ERP resource estimates; (3) EIA altered their conventional onshore natural gas production forecast as a result of an ERP study indicating that the predictions could not be met with domestic supply (4.5B); (4) the National Petroleum Council technology subgroup, tasked to determine if producing gas hydrates is feasible in the next 25 years, relies of ERP information and data (4.5C); (5) Geode was independently reviewed and found to have "significantly more functionality than ArcIMS software and is well positioned as an enterprise-level solution." (4.5D)

If the program includes construction of a facility, were program goals achieved within budgeted costs and established schedules?

Explanation:  

Evidence: