Detailed Information on the
USDA Wildland Fire Management Assessment

Refine program delivery by basing allocations of program resource on the basis of risk mitigation, emphasizing accountability for firefighting costs, improving management oversight, and ensuring fair sharing of costs. These refinements will be reflected in a national strategy for aligning incentives, improving accountability, and controlling costs by allocating resources on the basis of risk. This strategy will outline internal controls to ensure overall spending--especially for wildfire suppression actions--occurs within constraints; address organizational and structural changes, if necessary; align preparedness and suppression resources to accurately reflect program roles and responsibilities; and specify a consistent analytical process for allocating resources based upon assessed risk levels. In doing so, the strategy will also address specific deliverables, timeframes, and accountable personnel. The Forest Service will also undertake other such actions as necessary to implement the desired outcomes of this plan.

Improve performance data reliability to fully evaluate progress in reducing risk of catastrophic fire. This includes both program performance data and management indicators, and involves Fire Program Analysis, LANDFIRE, the National Fire Plan Operations and Reporting System, financial management, and other systems used by the program. Reporting is to include detailed information, such as breaking down accomplishments by region, noting changes in condition class, and differentiating between initial and maintenance treatments and multiple treatments on the same acres. The assignments of targets and the accountability of agency officials will be based on this improved data. The Forest Service will also undertake other such actions as necessary to implement the desired outcomes of this plan.

Improve procedures for allocating hazardous fuels reduction funds by assessing the risks from wildland fires and determining the benefits of fuels treatment and restoration projects by priority. In doing so, the Forest Service will implement recommendations contained in relevant Government Accountability Office and Office of Inspector General to ensure that the most important and highest priority projects are funded first. The Forest Service will also undertake other such actions as necessary to implement the desired outcomes of this plan.

Measure: Percentage of total National Forest System land base for which fire risk is reduced though movement to a better condition class.

Explanation:This measure provides an indication of the percent increase of all National Forest System (NFS) lands moved into good condition (defined as acres in condition class 1) and reflects the percent of high priority acres where fire management objectives are achieved. This measure will enable the Forest Service to measure changes in the landscape over time that reduce the overall fire risk. Fire Regime Condition Class (FRCC) data has been collected since 2002. However, the Forest Service only recently began including other activities (such as improving wildlife habitat) that also meet fire objectives in the annual accomplishment for acres treated to reduce hazardous fuel. Therefore, the baseline from 2005 is more reliable than the 2002 baseline data which only includes FRCC. While the target percentage increase from year to year appears small, due to the size of the 193 million acres NFS, the increase in the number of acres treated is significant. Further, target accomplishments are cumulative.

Measure: Percent of fires not contained in initial attack that exceed a Stratified Cost Index.

Explanation:The stratified cost index is a measure that compares actual expenditures on large wildland fires to what one would expect the fire to cost given certain characteristics, such as size, fuel type, or proximity to towns. It will enable the Forest Service to (1) determine which fires exceeded an acceptable normal range, (2) identify the reasons that occurred and learn from them and (3) better hold senior officials at large fires accountable for controlling costs. Fires that "exceed" the index are those with costs higher than one standard deviation from the acceptable norm for that type of fire.

Measure: Number of acres maintained and improved by treatment category (prescribed fire, mechanical, and wildland fire use) and of those improved the percent that change condition class.

Explanation:This measure allows the Forest Service to receive credit for all of the resource management actions done to reduce fire risk while accurately portraying the progress made in achieving the ultimate goal, which is a change in condition class. The baseline will be established for FY 2008 based on the results of the Hazardous Fuels Prioritization and Allocation System. The system will identify those Forest Service units that are high priority for funding based on risk of catastrophic fire events (Hazard, Probability, Consequence), unit efficiency, and ecological considerations.

Measure: Percent change from the 10 year average for (1) number of wildfires controlled during initial attack and (2) number of human caused wildfires.

Explanation:This measure will allow the Forest Service to gauge its progress in achieving its annual goals as compared to its long-term, historical progress. The 2005 Baseline is the 10 year average from 1996-2005. Initial attack is the action taken by resources that are first to arrive at an incident. All wildland fires that are controlled by suppression forces undergo initial attack. To ensure the Forest Service continues to improve performance each year the baseline will be a rolling 10 year average. As the Forest Service accomplishes program goals the average should go down each year. The targets for the measure are (1) not more than a 0.5 percent reduction from the 10 year average for the number of fires controlled during initial attack (the aim is to hold this number steady or increase annual accomplishments) and (2) two percent annual reduction from the 10 year average for the number of human caused wildfires.

Measure: Total acres treated in Wildland Urban Interface (WUI) and non-WUI and also acres treated for other vegetation management activities that achieved fire objectives as a secondary benefit.

Explanation:This measure tracks the number of acres treated inside and outside the Wildland Urban Interface as well as other land management activites that contribute to the reduction of fire risk. This measure will be used pending implementation of a measure of the number of acres maintained and improved by treatment category (prescribed fire, mechanical, and wildland fire use) and of those improved the percent that change condition class, currently under development.

Measure: Number of acres restored and maintained per million dollars gross investment.

Explanation:This measure enables the Forest Service to better monitor and track all hazardous fuels treatments in the context of the program's ultimate desired end result??change in condition class that results in reduced risk. Baseline for this new measure to be established for FY 2008 based on the results of the Hazardous Fuels Prioritization and Allocation System. The system will identify those Forest Service units that are high priority for funding based on risk of catastrophic fire events (Hazard, Probability, Consequence), unit efficiency, and ecological considerations.

Measure: Acres moved to a better condition class per million dollars gross investment.

Explanation:

Is the program purpose clear?

Explanation: The Wildland Fire Management program is responsible for protecting life, property, and natural resources on the 193 million acres of National Forest System (NFS) land and an additional 20 million acres of adjacent State and private lands covered by reciprocal protection agreements. The Forest Service accomplishes this by managing and, where appropriate, extinguishing wildland fires. The Forest Service has 4 specific goals for managing wildfire as defined in the joint Department of Agriculture (USDA) and Department of the Interior (DOI) 10-Year Comprehensive Strategy and Implementation Plan: (1) Improve Fire Prevention and Suppression ; (2) Reduce Hazardous Fuels; (3) Restore Fire-Adapted Ecosystems; and (4) Promote Community Assistance. While clear, these goals are set out as co-equal and would be improved if the Forest Service established clearer priorities. For example, the Forest Service in general puts reducing threats to human life and property as the highest priority and minimizing and controlling wildfire in remote, unpopulated areas lowest. However, program goals do not reflect this priority. Firefighting and fuels reduction resources can be directed at two different purposes: (1) protecting homes and buildings and (2) protecting natural resources. In some areas, both purposes are met with the same resources, but in other circumstances these two purposes may compete for limited resources. Determining the appropriate balance between these purposes can be subjective and complex. As a result, funds may not be targeted in the most cost-effective manner.

Evidence: Various authorization acts cited annually in the Wildland Fire Congressional Budget Justifications. The National Fire Plan (August 2000 Report to the President and the FY 2001 Appropriation Act and Reports) 10-Year Comprehensive Strategy (August 2001) and Implementation Plan (May 2002). The 1995 Federal Fire Policy and the 2001 update of that policy promote the re-establishment of fire adapted ecosystems to reduce large fires and protect communities. The President's Healthy Forests Initiative (2002) and the Healthy Forests Restoration Act of 2003.

Does the program address a specific and existing problem, interest, or need?

Explanation: Wildfires are normal events in the life of a forest and other wildlands. However, they can inflict damage to lives, property, and, in some cases, natural resources when they burn out of control, so the program addresses a specifc and ongoing need. The Forest Service wildland firefighting program is challenged with trying to balance allowing fire to function in its natural role with protecting communities, and in some cases ecosystems, from the effects of catastrophic fire when fire fuels have built up to unnatural levels. The program provides skilled firefighters, specialized equipment, and rapid response capability to respond to wildland fires in a way that many states and localities would be incapable of providing themselves. This capability is not matched by any other firefighting organization. Fire management on federal lands also falls within the land-management missions of the federal firefighting agencies. Without the Forest Service wildland fire management program, many communities would be unequipped to protect their own lands and property.

Evidence: 16 U.S.C. 551 (directs the Secretary of Agriculture to make provisions for the protection against destruction by fire and depredations upon the public forest and national forests). P.L. 101-121, The Department of Interior and Related Agencies Appropriations Act of 1990 (established a new appropriation for necessary expenses of firefighting, presuppression, and fuels management).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: As part of its land management responsibilities, the Forest Service is responsible for firefighting on the National Forests and Grasslands. There are no other Federal, state or local programs designed to accomplish this mission on Forest Service lands. The Forest Service works cooperatively with other Federal and state agencies to leverage funds and resources to extend the effectiveness of the firefighting program. However, these programs are complimentary and not redundant. Under these arrangements, Federal and state fire protection agencies cooperate and share scarce resources to provide mutual aid. In addition, the Forest Service is responsible for planning and carrying out hazardous fuel treatments on its lands but works cooperatively with other Federal and state agencies to improve coordination and more efficiently and effectively use resources to reduce the risk of fires before they occur. The Forest Service also funds an array of national firefighting and supports assets that are made available to other Federal agencies and state governments.

Evidence: 2001 Federal Wildland Fire Policy; National Fire Plan ("A Report to the President in Response to the Wildfires of 2000, September 8, 2000"); Interior and Related Agencies Appropriation Act 2006; Federal Wildland Fire Leadership Council Charter; Interagency Strategy for the Implementation of the Federal Wildland Fire Management Policy (June 20, 3002)

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that an alternative form of program design such as regulation would be more effective in fighting fires and reducing fire risks to communities and the environment. The Forest Service aligns the program's budget structure between program activities and funding sources, and activities are expected to be coordinated with other agencies (such as the Department of the Interior), state and local governments, and other program partners. Whether program resources are optimally allocated among (and efficiently used within) the various components of the program is an open question, but the design of the program, with its holistic approach to managing fire risk, appears to be appropriate.

Evidence: National Interagency Mobilization Guide (Chapters 10 and 20)(March 2005); Master Fire Agreement between BLM, BIA, NPS, FWS, and the USFS including regional and local agreements that tier to the Master; Border Agreements with Canada and Mexico; Fire suppression agreement with Australia and New Zealand; Interagency Standards for Fire and Fire Aviation Operations (the "Red Book") (January 2005)

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The beneficiaries for this program include communities protected from uncontrolled wildfire and the natural resources that constitute fire-adapted ecosystems. The Forest Service targets resources to reach these beneficiaries by establishing agreements with program partners (Federal, State, Tribal and local) that efficiently share suppression and other fire personnel and equipment as needed to accomplish program goals. This structure may in some cases provide resources that state or local governments should supply, but the agency's use of seasonal employees, contract aircraft, emergency firefighters, and partnerships with state, tribal and local governments, the military, and foreign nations, allows it to meet its primary responsibility to conduct wildland firefighting on the national forests. Further, the use of highly mobile national resources such as hotshot crews, smokejumpers, helicopters, and airtankers are used where necessary to extend local or regional firefighting capabilities to reach intended beneficiaries while reducing the risk of unitended subsidies.

Evidence: FY 2006 FS Budget Justification (Wildland Fire Management); National Fire Plan ("A Report to the President in Response to the Wildfires of 2000, September 8, 2000); Federal Wildland Fire Leadership Council Charter; National Interagency Mobilization Guide (Chapters 10 and 20)(March 2005); Master Fire Agreement between BLM, BIA, NPS, FWS, and the USFS including regional and local agreements that tier to the Master; Interagency Standards for Fire and Fire Aviation Operations (the "Red Book") (January 2005)

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Wildland Fire Program has three primary objectives including (1) reducing the risk of wildfires by reducing hazardous fuels, (2) preparing for potential wildfire occurrences and (3) suppressing wildfires. Each of these objectives reflect an intent to protect communities and natural resources while, as nearly as possible, allowing fire to function in its natural ecological role. The long-term outcome measures extend beyond 2008 and focus on ecological changes in the landscape over time resulting from program activities that reduce the overall fire risk in priority areas. Credible baseline data on Fire Regime Condition Class (FRCC) is available and used for this measure, but the data are at a scale and coarseness that limit the ability of the Forest Service and others to use FRCC for on-the-ground decisionmaking. The Forest Service and its partners are working on the development of a national fire risk rating measure, a nationally consistent and locally relevant metric.

Evidence: 10-Year Comprehensive Strategy Implementation Plan Update (2006); FRCC baseline data

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has no targets or timeframes established for its long-term measure presented in annual budget justifications. This is primarily due the lack of sufficiently accurate"on-the-ground" data. The program does have a clear baseline and that shows progress in meeting long-term goals, however. The Forest Service is developing a method for weighing risks against consequences to determine the highest geographic priorities, which in turn could lead to specific quantified targets and timeframes to accomplish those targets. This method relies on data that has only recently become available.

Evidence: NFPORS; USDA Forest Service Strategic Plan for Fiscal Years 2004-08; FY 2006 Forest Service Budget Program Direction, Chapter 3; Development of Coarse-Scale Spatial Data for Wildland Fire and Fuel management. Gen. Tech. Rep. RMRS-GTR-87 (2002).

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The "yes" answer for this question is based on an acknowledgement that there are still significant improvements needed in this area and that Forest Service is committing to making these improvements in the near term in conjunction with the current review of the 10-Year Implementation Plan in the light of new understanding and practical experience with existing measures. The Forest Service has one suitable annual output measure for the program and an appropriate efficiency measure. The annual output measure, the number and percent of acres improved by treatment category (mechanical, prescribed fire, and wildland fire use), and of those improved the number and percent that improved condition class, links annual management actions with the long-term outcome measure that focuses on ecological changes in the landscape over time. The annual efficiency measure consists of the percent of fires not contained in initial attack that exceed a stratified cost index and helps the Forest Service balance firefighting actions that protect communities and natural resources with efforts that would allow fire to function in its natural ecological role. However, a new annual output measure of the change in the ten-year average for the number of acres burned by wildfire may serve as an incentive to suppress more fires than necessary and thereby perpetuate the ecological inbalances the program seeks to restore. A new annual efficiency measure focuses on program investment in the Wildland Urban Interface (WUI), but the Forest Service has neither defined the limits of nor quantified the extent of the WUI. The Forest Service needs to do more work on these metrics.

Evidence: 10-Year Comprehensive Strategy Implementation Plan Update

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has set baselines and targets for its annual measures. Baselines exist in the form of actual accomplishments for the first year each measure was reported. Targets are revised annually to reflect changes in resource levels, anticipated fuels and weather conditions, and growing knowledge about wildland fire behavior. These targets focus on increasing the percentage of acres treated for hazardous fuels reduction where the risks to life and property are greatest, maintaining a high success rate in containing wildfires before they become large and damaging, and improving the fire-related conditions in wildlands. The Forest Service is steadily increasing both its performance and targets for the measures that are most indicative of progress toward achieving long-term outcomes. The baseline data for all of the annual measures, except the percent of fires that exceed a stratified cost index, has been collected in the agency's NFPORS database since 2002. The stratified cost index is a new efficiency measure and its baseline data was developed in 2005 using 10 years of historical data.

Evidence: FY 2006 Forest Service Budget Program Direction, Chapter 3; USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Rocky Mountain Research Station Data; NFPORS database

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Partners include state, local, and private partners responsible for fuels reduction work on adjacent private or state lands. While many partners (e.g., the National Association of State Foresters, National Association of Counties, and Western Governor's Association) demonstrate broadly a commitment to the program's goals, the 10-Year Comprehensive Strategy Implementation Plan included performance measures that were to be reported on by all parties. The agencies do not report this information, so there is no way to assess the extent to which States, Tribes, and local partners are actually reporting this performance information or meeting their responsibilities under the 10-Year Plan. Much work remains to be done in actually collecting meaningful performance information for work done on state and private lands. It is also difficult to make the case that all partners are able to support program planning efforts by committing to the goals of the program because the program received a a qualified "Yes" on Question 2.3. The Forest Service indicates that the agencies are working to improve reporting of performance and that the Western Governor's Association is working to develop reporting requirements and mechanisms for the States. The program does have cooperative agreements or MOU's on fuels treatment development and collaboration with many partners, including the National Association of State Foresters, the National Association of Counties, the Nature Conservancy, the International Association of Fire Chiefs, and the National Association of Conservation Districts. Many individual projects are developed and conducted with conservation organizations including the Rocky Mountain Elk Foundation, Wild Turkey Federation, Mule Deer Foundation, etc. Contractors are required to report on performance (e.g., fuels reduction projects), and grants are monitored to ensure that the grantee (e.g., a local firefighting unit) uses the funds as specified.

Evidence: Wildland Fire Management Congressional Budget Justifications, 2004-2007; 2001 Federal Wildland Fire Management Policy; USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Ten-year Comprehensive Strategy, Implementation Plan; NFPORS Database Accomplishments; Results-Oriented Government: Practices That Can Help Enhance and Sustain Collaboration among Federal Agencies, GAO-06-15, October 21, 2005.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: There have been many independent evaluations, conducted on a regular basis, that address program improvements and evaluate effectiveness. A number of independent reviews have occured or are currently ongoing, but no regular process has been established. The GAO, USDA OIG, NAPA, and other independent organizations have looked at various components of the wildland fire program including fuel treatments, suppression costs, safety, incident management and overall program management. While the scope of each of these reviews may be somewhat limited, all of the reviews taken together provide a fairly complete picture of the overall management of the program. In addition, all of these external reviews are independent and unbiased and GAO and OIG conduct rigorous reviews in accordance with Generally Accepted Government Auditing Standards. Since 2004, the Secretary of Agriculture has been appointing an independent review panel to study costs of fires that exceed $10 million. There are no constraints placed on the independent panel, and they report directly to the Secretary of Agriculture on their findings. The Forest Service used the findings and recommendations from these reports to improve management oversight, enhance safety, and improve delivery of program services.

Evidence: Wildland Fire Management: Important Progress Has Been Made, but Challenges Remain to Completing a Cohesive Strategy, GAO-05-147, January 14, 2005 (http://www.gao.gov/new.items/d05147.pdf); Independent Large Fire Cost Reviews; GAO; OIG; NAPA; TriData

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Forest Service cannot ensure that the accomplishment information it provides to Congress and other interested parties is consistent, valid, and supported, rendering it largely ineffective in managing Forest Service operations. Also, the Forest Service's Performance and Accountability Report was based on inaccurate, inconsistent, and unverified data, and thus unreliable for making sound decisions. Consequently, resource allocations do not reflect desired performance levels nor are the effects of funding and other policy changes clear. The Forest Service has made some improvements in its planning processes but has not effectively implemented these improvements. Further, the Forest Service does not clearly explain the relationship between budget resources and performance levels, with the possible exception of the hazardous fuels reduction program component. However, even for fuels reduction, the Forest Service can only provide a reasonable relationship to annual goals for acres treated. There remains no context within which to assess how different funding levels will impact the program's ability to meet its broad long-term goals of reducing risks to communities and the environment, although the Forest Service is working on the development of a risk-based allocation methodology that can improve agency attainment of performance goals.

Evidence: USDA Office of Inspector General Audit Report No. 08601-1-Hy, Forest Service Implementation of the Government Performance and Results Act, found that standards defining performance vary between regions and national forests and even among the districts within a single national forest. See also: USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Forest Service FY 2004 - FY 2006 Budget Justifications; FY 2007 USDA Forest Service Budget Justification (http://www.fs.fed.us/aboutus/budget/); FY 2004 - FY 2006 annual program directions.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Forest Service has taken a number of steps to improve its strategic planning, although much work remains to be done. For its aviation fleet, the costs for which currently run into the hundreds of millions of dollars annually, the Forest Service has not developed a capital asset plan for the full life-cycle costs, including acquisition, operation, and maintenance. The agency does not currently report on 8 of the 18 measures contained in the 10-Year Implementation Plan, including the only measure of fire suppression cost effectiveness. It also does not collect performance information from state and local partners for activities these partners undertake on their own lands. Further, neither the Forest Service nor the Department of the Interior (DOI) have articulated long-term outcome targets for the program, nor have they adequately defined the wildland-urban interface to foster prioritization of available resources. However, in 2005 the Forest Service and its partner federal agencies conducted a Quadrennial Fire and Fuel Review patterned after the quadrennial Defense Review. It provides a look at national demographic and climatological changes affecting wildland fire management and sets forth a strategic vision for the federal wildland fire agencies. The agency recently worked with its partners to revise the 10-Year Comprehensive Strategy Implementation Plan, and has developed the National Fire Plan Operations and Reporting System (NFPORS) to collect the underlying performance data for many of these measures. DOI and USDA have also been developing the Fire Program Analysis (FPA) system, which should eventually provide an economically sound method of allocating resources (i.e., people and equipment) in the most cost-effective manner at any given budget level. Similarly, once it is fully operational, the joint LANDFIRE system is expected to provide over 20 spatial data layers for fire and land managers to use to target fire and resource management projects most effectively.

Evidence: "Forest Service Procurement of Lead Planes," USDA Office of Inspector General, Report USDA/OIG-A/08601-37-SF (2004) found the Forest Service did not conduct a required analysis of its aviation procurement plans. See also: 10-Year Comprehensive Strategy Implementation Plan; Quadrennial Fire and Fuel Review Report (2005).

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Forest Service and DOI have established a standard automated data collection system (NFPORS) for reporting accomplishments. Although NFPORS has significantly improved the agency's collection of performance information, it has become clear that there still remain substantial limitations in the data collected. For example, the agency does not collect information on how wildland-urban interface (WUI) treatments improved the fire fuels condition of the acres treated. This is increasingly important as the program emphasizes WUI treatments over non-WUI treatments. The Forest Service reports its leaders and managers conduct monthly meetings to discuss regional capacity to achieve their goals. However, these meetings appear to focus on whether the regions can meet assigned targets of the number of acres each region is to accomplishments rather than using performance data to assess progress toward the program goal of reducing fire risk. They also appear to address other factors not directly related to fire risk outcome goals such as treatment type (mechanical means vs. prescribed fire) or by whom the acreage treatment is accomplished (whether the work was completed in-house or by contract).

Evidence: FY06 Budget Program Direction, Chapter 3; National Fire Operations Reporting System (NFPORS) Charter

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Forest Service reports its program managers are issued targets for hazardous fuels reduction other vegetation management programs bases individual performance ratings on accomplishment of these targets, it has not provided evidence of how performance is incorporated into personnel reviews outside of the fuels program component. Further, the Forest Service has not provided evidence of how performance is incorporated into personnel reviews outside of the fuels program component. Indeed, an independent audit found that the USDA Forest Service's Pacific Southwest had obligated funds in excess of its sub-allocation by approximately $4.0 million at the end of FY 2005, apparently contrary to prohibitions contained in the Anti-Deficiency Act, but neither regional line officers nor program managers appear to have been held accountable for spending more funds than they were allocated. On the firefighting side, managers are currently evaluated based on defacto performance measures of controlling fires without the loss of life or property. If the cost of the fire exceeds $10 million, an independent review panel conducts an evaluation and reports to the Secretary of Agriculture on their findings. The Forest Service indicates that suppression cost-control has been elevated to a significant factor in determining if fire response was appropriate to the threats posed by fires, but it is not clear that managers have actually been held accountable for excessive costs. However, a new performance measure will use a Stratified Cost Index (SCI) that will enable the Forest Service to evaluate the cost and efficiency of management on individual large fires as compared with historical costs on fires with similar characteristics. The measure will serve as an indicator of managers' performance and accountability relative to large fire cost.

Evidence: FY 2004 and FY 2005 (08401-6-FM) Forest Service Audit Reports from KPMG LLP and reviewed by USDA Office of Inspector General (http://www.usda.gov/oig/rptsauditsfs.htm). NFPORS database and performance data; FY 2006 Program Direction; Stratified Cost Index Report.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The report of an independent auditor on the Forest Service's FY 2004 and FY 2005 Financial Statements stated the "USDA Forest Service does not obligate all transactions required by appropriations law" as well as its belief that the USDA Forest Service may not be in compliance with the Anti-Deficiency Act, 31 U.S.C. 1517. Other systems appear to be functioning well, however. On large fires with Type 1 or Type 11 Management Teams the fire costs (obligations) are being collected using the Incident Cost Accounting & Accounting System (ICARS) beginning within 72 hours and continuing until the fire is controlled. Small fires up to 500 acres have an accrual processed in the accounting system at the end of the FY if an individual vendor has obligations $25,000 or more. All large fires exceeding expenditures of $5 million are reviewed, in part, for obligations and the appropriate use of those obligations. During fire season obligations are reviewed weekly and the analysis is provided to the Department and OMB. The agency now uses Integrated Acquisition System (IAS) and I-WEB, for partners. Obligations are automatically generated when purchases, contracts or agreements are approved in these systems. All obligations are reviewed quarterly for accuracy, timeliness and intent. However, problems associated with billing procedures have affected the timely and accurate report of program awards.

Evidence: In its Forest Service Financial Statements for Fiscal Years 2005 and 2004 (Report No. 08401-5-FM, November, 2005), the independent auditor Obligation testwork performed over approximately 132 transactions disclosed that about twenty percent of tested transactions were not obligated as required by appropriation law prior to payment, including temporary travel, GSA automobile leases, and probable contingent liability type transactions. It also reported that to maintain administrative control of funds, the USDA Forest Service made sub-allocations to its organizational components, and stated that at the end of FY 2005, they understood that the USDA Forest Service's Region 5 had obligated funds in excess of its sub-allocation by approximately $4.0 million. The Forest Service is working to resolve these issues. Circular A-123 internal control narratives and control matrix; Quarterly SF 133 reports - 2003-2005; Fire Annual Operating Plan; Large Fire Review Plan; Incident Obligation Direction.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: There is little evidence demonstrating that the program has such procedures in place. The agency has not yet performed systematic or formal cost comparisons for positions within this program, although the agency is assessing the feasibility of new competitive sourcing studies for 2007 and 2008. The Department continues to emphasize contracting for hazardous fuels reduction projects. However, the agency has not completed any analysis to indicate that the use of contracting is more efficient than in-house resources in achieving fuels reduction goals, and the emphasis seems to be guided more by Congressional direction to hire local contractors than in ensuring the most efficient and effective use of resources to achieve the program's objectives. The agency does not report performance on the only measure of fire suppression cost effectiveness included in the 10-Year Implementation Plan. Fire suppression costs have increased substantially in recent years, and the agency currently does not appear to have effective procedures in place to measure and achieve efficiencies in fire suppression. Several efforts to address efficiencies are currently under way, however. Improvements have been made to the Wildland Fire Situation Analysis (WFSA) process to bolster cost-effective decision-making for large fires and a more comprehensive decision support system, the Wildland Fire Decision Support System, (WFDSS) is under development to enable cost effective decisions for fires that escape initial attack or are managed as wildland fire use events. The Forest Service will use a stratified cost index to provide an indication of cost effectiveness and will be the basis to guide further incident analysis to better understand cost factors and develop management strategies. Once fully implemented, these systems should provide the means to identify organizational efficiencies and determine the most cost effective allocation of fire protection resources.

Evidence: Wildland Fire Management Congressional Budget Justifications, 2004-2007; WFDSS project charter; Most Cost Effective brief; FPA Memorandum of Understanding; Decision Memo for NRE Undersecretary signature; Green Plan.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Forest Service has a longstanding practice of coordinating with the Department of the Interior (DOI) and State, tribal and local agencies in fire suppression efforts. GAO chose the Wildland Fire Management Program as an example of effective collaboration. The agencies also have taken a number of steps to increase coordination and collaboration both between DOI and USDA and with state and local partners. Examples include: 1) development of the interagency NFPORS, FPA, and LANDFIRE systems; 2) development of the 10-Year Comprehensive Strategy and Implementation Plan; 3) creation of the Wildland Fire Leadership Council(WFLC); 4) consultation with program partners on preparation of state and local Community Wildfire Protection Plans (CWPPs) under the auspices of the Healthy Forests Restoration Act; and 5) coordinated WUI fuels treatment project selection. Suppression activities are managed on an interagency basis and integrate resources from DOI, the Forest Service, Tribes, and state and local governments. Shared resources include fire crews, engines, incident command, airtankers, helicopters, and smokejumpers. Coordination of local fuels treatment projects and prioritization is occurring between DOI, the States, tribal governments, and the Forest Service. State Fire Assistance programs coordinate and collaborate with Department of Interior land management agencies and the USDA Forest Service Fire Management program to improve fire suppression capability, prioritize hazardous fuel mitigation projects, and coordinate with Department of Homeland Security (FEMA) in grant selection and award.

Evidence: Memorandum of Understanding for the Development of a Collaborative Fuels Treatment Program among the USDA Forest Service, DOI, National Association of State Foresters and the National Association of Counties( January 2003); Memorandum of Understanding for Cooperation among Department of Interior, USDA Forest Service, Department of Agriculture, and Department of Homeland Security (January, 2006); Executive Integration Team Charter. National Integrated Review of Hazardous Fuels and the Restoration of Fire-Adapted Ecosystems; Coordinated budget direction for fuels and vegetation management programs; FY06 Program Direction; Results-Oriented Government: Practices That Can Help Enhance and Sustain Collaboration among Federal Agencies, GAO-06-15, October 21, 2005

Does the program use strong financial management practices?

Explanation: The program follows the Forest Service's financial management guidelines for committing, obligating, reprogramming, and reconciling appropriated funds. However, an independent auditor's report on FS' internal control structure over financial reporting identified five material internal control weaknesses, each of which has a direct relation to each program component. The auditor's report on Forest Service laws and regulations contains instances of noncompliance with appropriations law and instances of noncompliance with the Federal Financial Management Improvement Act of 1996. The same report also indicated the "USDA Forest Service does not obligate all transactions required by appropriations law" as well as its belief that the USDA Forest Service may not be in compliance with the Anti-Deficiency Act, 31 U.S.C. 1517.

Evidence: FY 2002 (08401-1-FM), FY 2002 and FY 2003 (08401-3-FM), FY 2003 and FY 2004 (08401-4-FM) and FY 2004 and FY 2005 (08401-6-FM) Forest Service Audit Reports from KPMG LLP and reviewed by USDA Office of Inspector General (http://www.usda.gov/oig/rptsauditsfs.htm). PART guidance states that agency-wide material weaknesses that have a direct relation to the program requires a "No." Guidance further states a program is eligible for a "Yes" only if there are no instances of non-compliance with laws and regulations related to financial management.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: As GAO has noted, the agencies have taken a number of positive steps to improve program management, but much work remains to be done. The Forest Service, working with the Department of the Interior (DOI), formed the Wildland Fire Leadership Council (WFLC), to solve problems in the wildland fire management program that come to their attention. A recent example is that WFLC has formed a Strategic Issues Panel to look at fire suppression costs. The Panel was formed by WFLC in response to concerns by Congress, agencies and the public that fire suppression costs have escalated to an unreasonable level. Based on the recommendations in this report, WFLC established Cost Action Teams to ensure implementation of the recommendations. The teams are scheduled to report their progress regularly. The agencies also developed the NFPORS system to collect certain project performance data, and established improved training and guidance to ensure firefighters are properly qualified. A number of changes have yet to be fully implemented, including development of a new fire preparedness planning model (FPA) and a system for tracking the change in fire fuels conditions over time (LANDFIRE). The agency could also improve management by improving its strategic planning. In addition, several other improvements are underway - FPA and LANDFIRE are in various stages of implementation and continued development. Each of these initiatives has an executive oversight committee to guide the project and address issues. FPA will provide a common interagency process to analyze cost effectiveness, including program oversight, across agency boundaries while LANDFIRE will provide common and consistent data to promote consistent project and program planning. The agencies need to continue to work to implement past PART recommendations, including: 1) improving accountability for fire suppression costs and ensuring that states are paying their fair share of such costs, and 2) improving the targeting of hazardous fuels treatments in the WUI. Ultimately, the agencies need to develop a strategy that integrates risk reduction principles by: 1) better defining the wildland-urban interface, 2) identifying better ways to measure performance for fuels treatments in the WUI, 3) analyzing and balancing the short-term risks of allowing increased wildland fire use with the short- and long-term benefits of reintroducing fire to fire-adapted ecosystems, and 4) linking the strategy to meaningful and achievable long-term performance goals. Another improvement includes enhancements to the Wildland Fire Situation Analysis (WFSA) process to support cost-effective decision making for large fires; a more comprehensive decision support system (the Wildland Fire Decision Support System, WFDSS) is under development. WFDSS, scheduled for completion in 2008, is a scaleable decision support tool that will help line officers make cost effective decisions for unplanned ignitions that escape initial attack or will be managed as wildland fire use events The WFDSS project scope includes re-engineering the existing Wildland Fire Situation Analysis (WFSA) and Wildland Fire Implementation Plan (WFIP) processes and supporting applications. Large fire costs are increasing as a result of many factors, and advances in fire behavior modeling, geospatial analysis, remote sensing (e.g. LANDFIRE), weather and climate forecasting, and other modeling tools can be leveraged to improve fire management decision-making.

Evidence: Wildland Fire Leadership Council Charter; FPA Memorandum of Understanding; LANDFIRE Charter/Briefing Paper; Integrated Program and Budget Direction; Latest FPA Update; WFDSS Briefing Paper (http://www.fs.fed.us/fire/wfsa/WFDSSBriefingPaperFinal.pdf); TriData Report. Wildland Fire Leadership Council Charter. Interagency Fire Program Management Qualifications Standards and Guide. 10-Year Comprehensive Strategy Implementation Plan, 2002. Wildland Fire Management Congressional Budget Justifications, 2004-2007. Development of the National Fire Plan Operating and Reporting System (NFPORS), Fire Program Analysis (FPA) system, and the LANDFIRE system. GAO Report 05-147: "Wildland Fire Management - Important Progress Has Been Made, but Challenges Remain to Completing a Cohesive Strategy", January 2005. GAO Report 04-705: "Wildland Fires: Forest Service and BLM Need Better Information and a Systematic Approach for Assessing the Risks of Environmental Effects", June 2004.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The long-term outcome measure uses Fire Regime Condition Class (FRCC) as a metric and the primary purpose behind the outcome is to reduce the risk of fire across the landscape. Recent year's data indicate that the Forest Service is increasing the number of acres upon which condition class improved. Importantly, however, because the program lacks ambitious targets, it is difficult to determine the adequacy of the Forest Service's progress in amount of risk that has been reduced across the landscape. The recent development of a risk-based hazardous fuels allocation meothodology should allow the Forest Service to better refine and focus limited fuel treatment efforts and resources in high priority areas. Upon the establishment of ambitious targets, this should improve the program's ability to measure the long-term progress on the landscape of reducing the overall fire risk in priority areas.

Evidence: Wildland Fire Management Congressional Budget Justifications, 2004-2007. 10 Year Comprehensive Strategy Implementation Plan; NFPORS database accomplishments performance for the last four years; USDA Forest Service Strategic Plan for Fiscal Years 2004-08; FS FY 2005 and 2006 Annual Performance Plan; FY 2001 and 2002 National Fire Plan Annual Performance Report

Does the program (including program partners) achieve its annual performance goals?

Explanation: Results of annual performance measures indicate that the Forest Service is achieving many of its annual performance goals, but data quality questions and the definition of the wildland-urban interface lead to uncertainties regarding the extent of annual accomplishments. In FYs 2004 and 2005, the agency met or exceeded performance goals for the hazardous fuel reductions. In 2005 the agency had a target of 1,919 per million dollars gross investment. The actual agency accomplishment was 3,578 acres per million dollars gross investment.

Evidence: USDA Forest Service Strategic Plan for Fiscal Years 2004-08, Goal 1; Wildland Fire Leadership Council goals and strategies; FY 2006 Budget Justification; FY 2003-FY 2005 Performance and Accountability Reports

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Forest Service has instituted some new processes to attempt to contain suppression costs, but thus far has not demonstrated that these changes have had the desired impact on improving efficiencies. The new stratified cost index efficiency measure will let the program measure progress in containing wildland fire costs, but the measure has one year of baseline data only, so progress cannot yet be demonstrated. Once in place, this stratified cost index will enable the program to (1) determine which fires exceeded an acceptable normal range, (2) identify the reasons that occurred and learn from them and (3) better hold senior officials at large fires accountable for controlling costs. As discussed in 4.2 the Wildland Fire Program measures the Number of acres treated in condition classes 2 and 3 in fire regimes 1, 2, or 3 moved to a better condition class per million dollars gross investment. In 2004 the agency treated 2,946 acres per million and in 2005 it treated 3,578 acres per million dollars gross investment. However, this may be misleading since treatment costs per acre for wildland-urban interfrace (WUI) projects decreasing modestly while costs per acre for non-WUI projects are substantially increasing. This is despite the new flexibilities provided to the Forest Service through stewardship contracting and other new authorities under the Healthy Forests Restoration Act. One possibility for this apparent contradiction in the data is that it reflects the continuing problem with defining the "wildland urban interface." As the agencies increase their emphasis on WUI treatments over time, field staff and/or project proponents may simply be defining more projects as WUI projects in order to increase the likelihood of having their projects funded. Because non-WUI treatments are typically less expensive than WUI treatments, redefining non-WUI acres as WUI acres would tend to reduce the average per acre costs for WUI treatments.

Evidence: Cost sharing policy excerpt FSM 3100; Memorandum of Understanding for the Development of a Collaborative Fuels Treatment Program among the USDA Forest Service, DOI, National Association of State Foresters and the National Association of Counties( January 2003); FY 2007 Budget Justification; FY 2005 Performance and Accountability Report.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Department of the Interior (DOI) is the only other federal agency that accomplishes similar wildland fire management work. The Forest Service program is larger and provides a significant amount of contracted and agency resources to cooperators, including DOI. This adds complexity to the predictability of annual suppression expenditures for the Forest Service. In other areas of the fire program, performance of both agencies appears to be similar and complimentary. USDA and DOI are working on completing a common performance measure evaluation based upon their 10-year Strategy Implementation Plan.

Evidence: Wildland Fire Management performance tables in annual budget requests 2002-2007; Results-Oriented Government: Practices That Can Help Enhance and Sustain Collaboration among Federal Agencies, GAO-06-15, October 21, 2005

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Numerous GAO and other independent evaluations have been increasingly positive, over the last few years, of various parts of the fire program for the Forest Service and their partners. The GAO, USDA OIG, NAPA, and other independent organizations have looked at various components of the wildland fire program including fuel treatments, suppression costs, safety, and incident and overall program management. While these reviews are numerous, including GAO and OIG reviews that are independent, unbiased, and conducted in accordance with Generally Accepted Government Auditing Standards, the scope of each of these reviews has been somewhat limited and do not meet the standards required by PART guidance. In general, the reviews taken together provide a fairly complete picture of the overall management of the program, and have noted improvement in program management, and have made recommendations in areas where improvements are still needed. For example, USDA's Office of Inspector General found that the Forest Service's use of emergency equipment rental contracts requires improved contract oversight and better oversight of training. The reviews also point out that many changes have not yet been implemented and much work remains to ensuring that program is effective and achieving its long-term goals.

Evidence: Wildland Fire Management: Timely Identification of Long-Term Options and Funding Needs Is Critical, GAO-05-923T, July 14, 2005; Technology Assessment: Protecting Structures and Improving Communications during Wildland Fires, GAO-05-380, April 26, 2005; Wildland Fire Management: Important Progress Has Been Made, but Challenges Remain to Completing a Cohesive Strategy, GAO-05-147, January 14, 2005; Wildland Fires: Forest Service and BLM Need Better Information and a Systematic Approach for Assessing the Risks of Environmental Effects, GAO-04-705, June 24, 2004; Geospatial Information: Technologies Hold Promise for Wildland Fire Management, but Challenges Remain, GAO-03-1047, September 23, 2003; Wildland Fire Management: Additional Actions Required to Better Identify and Priorities Lands Needing Fuels Reduction, GAO-03-805, August 15, 2003; Forest Service: Information on Decisions Involving Fuels Reduction Activities, GAO-03-689R, May 14, 2003; Results-Oriented Government: Practices That Can Help Enhance and Sustain Collaboration among Federal Agencies, GAO-06-15, October 21, 2005; OIG Reports