Detailed Information on the
Forest Service: Watershed Assessment

Developing long-term outcome-based and efficiency measures and work to ensure cooperators link their proposed activities to the agency's Strategic Plan or annual performance measures. The Forest Service will also undertake other such actions as necessary to implement the desired outcomes of this plan.

Generating better water quality, habitat, and biological data useful to monitor and describe the condition and trend of aquatic resources for national and local application. In doing so, the National Forest System and State and Private Forestry deputy areas will work with Forest Service Research and Development on the development and implementation of comparable, scalable, statistically significant monitoring that reflects the effects of resource management activities. This data be used by the Forest Service to make informed decisions regarding program priorities, resource allocations, and general program management. Data will also be integrated into the forest planning process. The Forest Service will also undertake other such actions as necessary to implement the desired outcomes of this plan.

Develop and implement a strategy to prioritize watersheds for management activities as the basis for program allocations. The Forest Service will also undertake other such actions as necessary to implement the desired outcomes of this plan.

Measure: Acres of watershed improvement

Explanation:Watershed improvement activities include work done directly on the land to improve watershed condition so that they achieve Forest Plan goals and objectives and State water quality requirements.

Measure: Acres or miles of aquatic habitat enhanced to achieve desired ecological conditions as described in Forest Land Management Plans

Explanation:Acres of lake habitat and miles of stream habitat enhanced to achieve desired ecological conditions directly contribute to improve watershed conditions and to restore and maintain native and desired non-native plant and animal species diversity and reduce the rate of species endangerment by contributing to species recovery. Desired conditions are established by the Clean Water Act and the Endangered Species Act, (and other laws) and described in each Forest Plan.

Measure: Acres of terrestrial habitat enhanced to achieve desired ecological conditions described in Forest Land Management Plans

Explanation:Acres enhanced to achieve desired ecological conditions directly contribute to improve watershed conditions and to restore and maintain native and desired non-native plant and animal species diversity and reduce the rate of species endangerment by contributing to species recovery. Desired conditions are established by the Clean Water Act and the Endangered Species Act, (and other laws) and described in each Forest Plan.

Measure: Allotment acres and per cent administered to 100 percent of Forest Plan standards

Explanation:Controlling livestock grazing and maintaining adequate vegetation cover is essential to watershed health. Grazing allotments are considered successfully administered to standards when program personnel implement guidance and direction found in Forest Plans, Allotment Management Plans, Annual Operating Plans, annual operating instructions, grazing permits, agreements, and other documents.

Measure: Establish Forest Vegetation in Acres

Explanation:This measure includes acres funded by both annual appropriations and Restoration Trust funds.

Measure: Acres of non-industrial private forest (NIPF) land under approved stewardship management plans.

Explanation:The program provides natural resource planning assistance to NIPF landowners. State forestry personnel prepare or verify written Forest Stewardship Management Plans which identify actions to protect, maintain, restore and improve existing resources including soil, water, range aesthetic quality, recreation, timber and wildlife, in a manner that is compatible with landowner objectives. These plans must meet current National Program Standards and Guidelines. Each year, States/Territories report to the Forest Service the acres of NIPF land covered by newly approved Forest Stewardship Management Plans.

Measure: The number National Forest System impaired water segments removed from the EPA/States listing of impaired waters (CWA Section 303(d))

Explanation:The goal of the Forest Service is to reduce NFS impaired water segments by 25 percent by 2012. Currently, EPA reports 33,650 impaired water bodies nationally with 2,624 water bodies containing NFS lands which have been divided into 18,363 segments that contain at least 50 percent NFS lands. Waters not attaining State water quality standards are listed by individual States as "impaired" and reported to Congress every 2 years in the 305(b) report, or the "biennial water quality report to Congress." Each impaired segment requires the development of a watershed restoration action plan, called a "Total Maximum Daily Load" (TMDL) which specifies actions necessary to attain water quality standards. By law, EPA approves the listing of impaired waters and TMDLs. Some of the causes of water quality impaired on NFS lands are beyond the sole control of the agency and the Forest Service works cooperatively to resolve these issues to the extent feasible. However, not all impaired segments occurring on the National Forests can be resolved unilaterally by the agency. For example, water segments impaired due to acid deposition would required actions to be taken by multiple private and governmental entities. The proposed rate of reduction in the number of impaired segments is the same as the national target set by EPA's 2005 Surface Water Protection PART.

Is the program purpose clear?

Explanation: The Forest Service has a clear authority to manage a broad spectrum of watershed activities on the National Forests and to encourage the long-term stewardship of non-industrial private forestlands which contribute significantly to the health and productivity of the nation's watersheds. The purpose of the watershed program is to restore, enhance, and maintain watershed conditions including soil, water, air, and forest and rangeland vegetation within the national forests and grasslands. Management of these basic physical and biological resources provides the foundation for healthy, viable ecosystems and improves wildlife and fish habitat; range and forest stand conditions; water quality, quantity, and timing of stream flows; restoration of deteriorated watershed conditions due to past practices, and long-term maintenance of soil productivity.

Evidence: The Organic Administration Act of 1897 provides authority for the Forest Service to manage watersheds and secure favorable conditions for waterflow on national forests. The Multiple-Use Sustained-Yield Act of 1960 provides that national forests are to be administered for watershed and other purposes. Other laws relating to watersheds include: the Weeks Law of 1911; Knutson-Vandenberg Act (1930); National Forest Management Act of 1976; Section 323 of the Consolidated Appropriations Resolution Fiscal Year 2003 (PL. 108-7); the Restoration of National Forest Lands and Improvements Act (16 U.S.C. 579c); Section 6 of the Act of August 11, 1916 (Ch. 313, 39 Stat. 446, as amended; 16 U.S.C. 490); Section 327 of the Omnibus Consolidated Rescission and Appropriations Act of 1996 (P.L. 103-134); and the Recreational Boating Safety and Facilities Act of 1980 (P.L. 96-451, Title III, as amended). Watersheds are a strategic goal listed in the USDA Forest Service Strategic Plan for Fiscal Years 2004-08, and administrative guidance for watershed management is provided through Forest Service Manual Section 2500.

Does the program address a specific and existing problem, interest, or need?

Explanation: Sustaining national forest watersheds over time is an essential part of maintaining the ongoing productive capacity of the Nation's land and water. Today, 60 million people, or one-fifth of the Nation's population, draw source water from national forests and grasslands. An estimated 3,400 towns and cities currently depend on national forest watersheds for their public water supplies. In addition, privately owned forests are the source of nearly 30 percent of all freshwater flows. Although most National Forest System (NFS) watersheds are in satisfactory condition, many streams on NFS lands do not meet applicable water quality standards. Watershed management addresses these sources and standards. It also influences water quality, quantity, and timing of flows; forage conditions; forest health; fish and wildlife habitats; wildfires risks; and resistance to the establishment and spread of invasive species. Watershed research contributes to watershed condition by providing science-based validation of the effectiveness of current methods and techniques to improve watersheds.

Evidence: The role of forested watersheds in water supply??including quantity; quality; timing or release; flood reductions and low flow augmentation; and the economic value of water from national forests??was detailed in the report Water and the Forest Service, FS-660, January 2000. Long-term program goals are aligned with societal needs through the USDA Forest Service Strategic Plan for Fiscal Years 2004-08. Annual adjustments that reflect changing needs are formulated through the Budget (including the Forest Service FY 2006 Budget Justification) and prioritized in execution through the Forest Service FY 2006 Program Direction.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Forest Service is solely responsible for management of 193 million acres of National Forest System (NFS) lands. While other Federal or state entities establish regulatory standards concerning water, only the Forest Service is authorized to conduct watershed management activities on national forest land to meet those standards and ensure consistency with mission requirements. To limit redundancy, the Forest Service relies on water quantity data from other federal agencies, such as the U.S. Geological Survey. To avoid duplication with state efforts, the Forest Service provides watershed planning and other assistance to State Foresters or equivalent State officials to accomplish coordinated management of forest resources, including both national forest and non-Federal forest lands. Excessive overlap of research is avoided by Forest Service collaboration with partners with academic institutions and other federal research agencies, and the research of this program provides value beyond research conducted by other entities.

Evidence: The Organic Administration Act of 1897 provides authority for the Forest Service to manage watersheds and secure favorable conditions for waterflow. The Multiple-Use Sustained-Yield Act of 1960 provides that national forests are to be administered for watershed and other purposes. Authority to work with states is provided through the Cooperative Forestry Assistance Act of 1978. Management of national forests is delegated to the Chief of the Forest Service in Title 7, Code of Federal Regulations, Section 2.60. Administrative guidance for the conduct and coordination of Forest Service watershed management is provided through Forest Service Manual Sections 1200, 2500 and 3200.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that an alternative form of program design such as regulation would be more effective in meeting program goals, nor does the evidence indicate that structural changes are needed to increase efficiency or efficacy. The Forest Service is addressing needed program improvements such as performance measure data collection consistency through the program's current design. The Forest Service works with a large number of partners who contribute direct and indirect funds to improve conditions on forest and grasslands. These cooperative conservation partners leverage Federal funds to help achieve program goals within the current program design.

Evidence: The 1998 Committee of Scientists Report, "Sustaining the People's Land: Recommendations for Stewardship of the National Forests and Grasslands into the Next Century" (http://www.cof.orst.edu/org/scicomm/) recommended no major structural changes to the Forest Service. The Committee recognized that there is a national consensus on the importance of sustaining the lands and resources of the national forests and grasslands. They did not include recommendations to address program design flaws. An internal review of the program entitled "Rise to the Future Task Force 2003, Fisheries and Watershed Program" addressed programmatic and strategic objectives, accountability, and staffing. In doing so it identified no major design flaws. (Rise to the Future Task 2003 Force Report)

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Forest Service conducts no routine monitoring for water quality parameters except in the case of individual research projects. The Forest Service cannot amalgamate these disparate data in a scientifically valid manner to provide assessments of overall trends nor establish priorities to provide the most effective and efficient allocation of resources to meet the program's purpose and goals. As a result, the Forest Service is unable to target watershed improvement and restoration projects to the areas of greatest need within the allocation of available resources. While data quality remains an issue, the Forest Service is moving to address this need. For example, as part of the Northwest Forest Plan, the Forest Service recently completed "A Preliminary Assessment of the Condition of Watersheds" that presents an analytical approach for evaluating watershed condition that includes management attributes (e.g., road construction), stand condition, and in-channel monitoring data

Evidence: The Forest Service's water data quality practices were reviewed by the General Accountability Office in GAO-04-382, Watershed Management Better Coordination of Data Collection Efforts Needed to Support Key Decisions (June, 2004). GAO cited projects throughout the Forest Service that currently involve the collection of long-term data but found that no two shared identical objectives and, as a result, data collection methods vary based on individual project needs. In that same report, GAO found that Forest Service water quality data are currently stored in scattered internal databases, and a limited amount of data are stored in EPA's Storage and Retrieval (STORET) System. Recently, the agency has established a centralized data storage system that is now being implemented, but access to data varies, with some data available through the Internet and other data available by request in electronic or paper formats. Recognizing the need to improve field-level management alignment with issues of national scope and importance, an internal review (Rise to the Future Task 2003 Force Report) recommended the Forest Service clearly articulate national watershed program priorities. The watershed condition report is described in Pacific Northwest Research Station General Technical Report PNW-GTR-647, Northwest Forest Plan - The First 10 Years (1994-2003): Preliminary Assessment of the Condition of Watersheds (September 2005) http://www.fs.fed.us/pnw/publications/pnw_gtr647/; Community-Based Watershed Restoration Partnerships, Accomplishments for FY 2000-2002 (http://www.fs.fed.us/largewatershedprojects/)

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The watershed program lacks long-term outcome-based measures of the performance of land management activities on national forest watersheds or of demonstrated water quality improvement over time. Further, although the Forest Service strategic plan establishes a goal of increase the area of forest and grassland watersheds "in fully functional and productive condition," the watershed program does not have a measure that tracks overall watershed health and demonstrates the effects of watershed restoration activities. However, the Forest Service is making notable progress in the development of such a measure, as well as with the development of outcome measures for national forest watersheds, particularly through the development of the Best Management Practices (BMP) Evaluation Program, which will demonstrate the effectiveness of Forest Service implementation of BMPs to protect water quality and soil productivity. The number of impaired water segments on national forests that are removed from the EPA/States listing of impaired waters may serve as a mechanism that can be used to make inferences about watershed condition, but the basis for these inferences has not been clearly demonstrated. Also unclear is the relationship of annual outputs to long-term program outcomes, particularly in the areas of research, forest products, and grazing management. Measures are also lacking for Forest Service grant programs that attempt to improve the management of watersheds on non-Federal forest lands. However, the Forest Service is currently working with State partners to adopt well-defined, new measures by 2008 that will reflect the Forest Stewardship Program's long-term and cumulative impact with respect to intended resource management outcomes.

Evidence: PART Guidance defines outcome measure as describing the intended result of carrying out a program or activity. A review of Forest Service the FY 2004 - FY 2007 congressional justifications, the FY 2004 and FY 2005 USDA performance attainment reports (PARs) , the FY 2004 - FY 2006 annual program directions and other documents, found such measures do not exist for each element of the watershed program. This appears consistent with the USDA Office of Inspector General Audit Report No. 08601-1-Hy, Forest Service Implementation of the Government Performance and Results Act, which found that Forest Service goals and performance measures are often vague, open to varied interpretation, and not distributed to the field for timely implementation. Several contributing programs that compose the watershed program lack long-term outcome measures. Guidance provides that each of the programs included in the PART must receive a Yes to a given question in order to receive a Yes on that question. Further, no evidence was available that provided the methodology that demonstrates how outputs support outcomes in a logical fashion, as required by guidance.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Because the program lacks long-term outcome measures, targets and timeframes are not applicable.

Evidence: PART guidance requires a "No" for Question 2.2 if the answer to Question 2.1 is "No."

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Although the watershed program does not have specific long-term outcome performance measures that reflect the purpose of the program, annual output measures demonstrate performance in aquatic and terrestrial habitat restoration; establishment of forest lands; range allotment administered to standards; and forest planning on non-industrial private forest land. Annual ouput measure can be used to track performance since the condition of water quality is a direct function of activities within the watershed--that is, properly conducted forest management and grazing practices contributes to the maintenance of good watershed condition. These measures provide insight on the results of annual land management activities to improve water quality and reduce the effects of soil disturbances, key contributors toward the attainment of the goal of improved watershed condition.

Evidence: USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has established baselines and targets for most of its annual measures, but annual output targets decline at a rate faster than available resources. Over the upcoming year, the Forest Service should review progress and consider revising program targets.

Evidence: USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: On non-Federal watersheds, State Foresters are the Forest Service's primary partner, but most State Forester workplans do not link their proposed activities to the Forest Service's Strategic Plan or annual performance measures. Program allocations also do not link to the Forest Service's Strategic Plan. However, on national forests, the wildlife, fisheries, and watershed partnership program support the program goals and report accomplishment in furtherance of those goals. Forest Service Research and Development ensures commitment from academic and state organizational partners when establishing cooperative agreements that support the program.

Evidence: USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions. See also the USDA OIG Audit Report No. 08601-41-SF, Forest Service Collaborative Ventures and Partnerships with Non-Federal Entities and the Wildlife, Fish, Rare Plants Management System (WFRP-MS) partnership listing of fisheries, wildlife, and NatureWatch partners.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Independent reviews of Forest Service implementation and of the effectiveness of best management practices (BMPs) applied on National Forest System lands throughout the United States are done on a regular basis by many State water quality agencies. These independent reviews include both non-Federal and national forest lands, and the Forest Service usually receives high marks. Reviews measure water quality against regulated standards, and because they are scientifically valid, are proper in scope, independence, and regularity. Further, the National Council for Air and Stream Improvement, Inc., has also conducted separate non-point source surveys of the north central, southern, and western states to summarize BMP implementation and effectiveness in these areas.

Evidence: Rise to the Future 2003 Task Force Report; Programs Assessing Implementation and Effectiveness of State Forest Practice Rules and BMPs in the West by Ice et al., 2004; Forest Practices: Montana's Integrated Approach, pp. 36-38, (http://clfa.org/pdffiles/Montana.pdf); NCASI Technical Reports 686 (southern states) (http://www.ncasi.org//Publications/Detail.aspx?id=995); 706 (western states) (http://www.ncasi.org//Publications/Detail.aspx?id=1029), and 710 (north central states) (http://www.ncasi.org//Publications/Detail.aspx?id=1041); Forestry Best Management Practices for Water Quality Protection in Arkansas, Implementation Report, 2005 (http://www.forestry.state.ar.us/bmp/BMP_Report_2005.pdf); Results of Georgia's 2004 Silvicultural Best Management Practices Implementation and Compliance Survey, 2005 (http://www.gfc.state.ga.us/Resources/documents/SBMPICSurvey2004.pdf); Oregon Department of Forestry Forest Practices Compliance Monitoring Project: 1998 Pilot Study Results, 1999, pp. 4-6 (http://www.oregon.gov/ODF/PRIVATE_FORESTS/docs/fp/BMPPilotResultsTR5.pdf)

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Forest Service cannot ensure that the accomplishment information it provides to Congress and other interested parties is consistent, valid, and supported, rendering it largely ineffective in managing Forest Service operations. Also, the Forest Service's Performance and Accountability Report was based on inaccurate, inconsistent, and unverified data, and thus unreliable for making sound decisions. Consequently, resource allocations do not reflect desired performance levels nor are the effects of funding and other policy changes clear. The FS has made some improvements in its planning processes but has not effectively implemented these improvements.

Evidence: USDA Office of Inspector General Audit Report No. 08601-1-Hy, Forest Service Implementation of the Government Performance and Results Act, found that standards defining performance vary between regions and national forests and even among the districts within a single national forest. Further, without acceptable long-term measures and ambitious long-term annual targets, it is difficult to see how budgets can be linked to program goals. USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program is improving its outcome measures: Watershed outcome measures have evolved since the 2003 Forest Service Strategic Plan Update replacing the fully functioning watershed condition with improved, measurable outcomes. New corporate performance measures are being incorporated into the agency performance and accountability system. In cooperation with another federal agency, an outcome-based performance measure that uses the EPA 303(d) impaired water body list is in place. Additionally, the Forest Service is implementing a National Best Management Practices Effectiveness Evaluation Program that will measure how well practices applied to the land protect soil resources and water quality. Additional work is necessary for the cooperative grant portion of the program, however.

Evidence: USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions.

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: The Forest Service Research and Development staff conducts specific research on current known and potential watershed questions. The research program is accomplished through the agency-wide strategic planning and budget processes involving the Research Executive Team (FSRET). FSRET conducts annual assessments and compares potential benefits of Forest Service Research and Development (R&D) efforts against other similar research according to the nature and magnitude of current and anticipated issues and information needs required by land managers for effective natural resource management. The Forest Service compares research and development findings annually with other research scientists, who have validated the Forest Service program's credibility. Forest Service scientists also compare potential benefits against other programs when participating in professional organizations and related meetings. For example, watershed research scientists meet annually at interagency research forums and workshops with different federal, state, and local agencies, universities, and other organizations with interests related to water to analyze and compare potential benefits and research products in a regional forum that fosters independent verification.

Evidence: Science and Technology To Support Fresh Water Availability in the United States. National Science and Technology Council. Committee on Environment and Natural Resources. Subcommittee on Water Availability and Quality. 19 pages, National Academies Press, Washington D.C. 2004.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: Prioritization for the Watershed R&D program is accomplished through the R&D-wide strategic planning and budget processes involving the Research Executive Team (FSRET). Forest Service Research and Development (R&D) prioritizes research annually according to the nature and magnitude of current and anticipated issues and information needs required by land managers for effective natural resource management. Development of research priorities is based upon OMB R&D investment criteria for relevance and applicability; support of technology development needed to produce forest sustainability; and addressing of National Forest System priority science and information needs. The Forest Service annual program direction documents R&D priorities each year and includes watershed research as one of three national priorities. Watershed R&D priorities are linking Forest Service research watersheds with other agency's watersheds and data to promote the development of products that address large-scale land management issues needed to implement programs like the Healthy Forest Initiative and support Forest Health Monitoring-Intensive Site Monitoring needs. Requests and allocations reflect this process and these priorities.

Evidence: Forest Service Research Executive Team Directions to Staff and Stations; Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Although annual performance accomplishments are collected to report on annual program direction assigned targets, the Forest Service lacks statistically valid national water quality data. It also lacks comparable, statistically significant monitoring data between units of the National Forest System that reflects the effects of resource management activities. This lack of data severely hinders the ability of the Forest Service to make informed decisions regarding program priorities, resource allocations, and general program management. It also limits the Forest Service's ability to implement Forest Plans. The Forest Service is currently working to generate better water quality, habitat, and biological data useful to describe the condition and trend of aquatic resources that has potential for national application. For example, the program has developed a system to manage, monitor, and report program accomplishments and opportunities at the district, forest, regional, and national levels. In 2006 Research and Development (R&D) is having an external organization evaluate customer satisfaction with R&D tools.

Evidence: General Accountability Office report GAO-04-382, Watershed Management Better Coordination of Data Collection Efforts Needed to Support Key Decisions (June, 2004). USDA Office of Inspector General Audit Report No. 08601-1-Hy, Forest Service Implementation of the Government Performance and Results Act. Forest Service Performance and Accountability Reports for FY 2004 and FY 2005; WORKPLAN (agency work budget and planning database); FY 2006 agency program direction to guide obligations project performance measures.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: All agency line officer (Regional Foresters, Forest Supervisors and District Rangers) performance standards and evaluation criteria include requirements in support of national performance measures related to agency strategic goals and objectives and assigned annual performance plan targets. Agency managers are accountable for costs, schedules and performance results as documented in the project budget and performance accountability WORKPLAN data system and in performance evaluations that assess performance under GPRA strategic goals, including improving watershed conditions. Accomplishments are reported and tracked through WORKPLAN and beginning in FY 2007, reported through the performance and accountability system (PAS) database. R&D line officer performance standards and evaluation criteria include requirements on GPRA goals, and the attainment of annual performance targets. Cost, schedule, and results of research work units are evaluated every 5 years. Challenge cost-share agreement financial plans are monitored for performance results prior to paying invoices for products and services. R&D cooperative agreement instruments outline performance standards and require periodic status reporting in order to obtain funding.

Evidence: Line Officer Annual Evaluations and Performance Standards; Challenge cost-share agreement exemplar.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The report of an independent auditor on the Forest Service's FY 2004 and FY 2005 Financial Statements stated the "USDA Forest Service does not obligate all transactions required by appropriations law" as well as its belief that the USDA Forest Service may not be in compliance with the Anti-Deficiency Act, 31 U.S.C. 1517. Other systems appear to be functioning well, however. For example, Federal and partner funds are obligated in a timely manner and reviewed on a quarterly basis for accuracy, timeliness, and intent. These process will be reviewed as part of the Forest Service's internal controls as prescribed by Circular A-123. Forest Service policy requires a project to charge expenditures to the fund code appropriated by Congress for the purpose of achieving the goals of the intended project. Additionally, the WorkPlan system requires units to plan every project, including identifying the funds that will be used, and links the project to the established performance measure associated the funds chosen, but not all of these measures meet PART standards.

Evidence: In its Forest Service Financial Statements for Fiscal Years 2005 and 2004 (Report No. 08401-5-FM, November, 2005), the independent auditor Obligation testwork performed over approximately 132 transactions disclosed that about twenty percent of tested transactions were not obligated as required by appropriation law prior to payment, including temporary travel, GSA automobile leases, and probable contingent liability type transactions. It also reported that to maintain administrative control of funds, the USDA Forest Service made sub-allocations to its organizational components, and stated that at the end of FY 2005, they understood that the USDA Forest Service's Region 5 had obligated funds in excess of its sub-allocation by approximately $4.0 million. The Forest Service is working to resolve these issues. Circular A-123 internal control narratives and control matrix; FY 2006 Program Direction; Audit Report, Forest Service FY 2004 and FY 2005 Financial Statements.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not include an appropriate efficiency measure, although the program effectively leverages the use of volunteers.

Evidence: USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions. See also the USDA OIG Audit Report No. 08601-41-SF, Forest Service Collaborative Ventures and Partnerships with Non-Federal Entities and the Wildlife, Fish, Rare Plants Management System (WFRP-MS) partnership listing of fisheries, wildlife, and NatureWatch partners. PART guidance requires efficiency measures to track the ratio of total outcomes or outputs to total inputs, which can include time or funds. Guidance states that "(l)everaging program resources can be a rationale policy decision, as it leads to risk or cost sharing; however, it is not an acceptable efficiency measure, because the leveraging ratio of non-Federal to Federal dollars represents only inputs." Further, the Forest Service has neither subjected the program to competition nor provided evidence of inter-agency information technology and monitoring data coordination with EPA or Federal or State land management agencies.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Close collaboration among programs within and external to the Forest Service is critical to program success and contributes to management and resource allocation actions. The program coordinates and collaborates with numerous other Federal, State, local agencies, and private entities to establish and fulfill management performance targets. For example, the program coordinates with EPA and State water quality and air quality to establish performance targets relating to the Clean Water Act and Clean Air Act. Similarly, the program coordinates with the Fish and Wildlife Service, National Marine Fisheries Service, and State wildlife management agencies protect habitat for species listed as threatened or endangered under the Endangered Species Act requires.

Evidence: Examples of coordination and collaboration include the Community-Based Watershed Restoration Partnerships consisting of 16 large-scale watershed restoration projects located in 26 states, Community-Based Watershed Restoration Partnerships, Accomplishments for FY 2000-2002 (http://www.fs.fed.us/largewatershedprojects/) and the development of a regional protocol for monitoring the application and effectiveness of best management practices, A Protocol for Monitoring Best Management Practices, Northeast Area, S&PF 2005 (http://www.na.fs.fed.us/aboutus/SpecialInitiatives/bestmgmtpractices/bestmgmtpractices05.htm).

Does the program use strong financial management practices?

Explanation: The program follows the Forest Service's financial management guidelines for committing, obligating, reprogramming, and reconciling appropriated funds. However, an independent auditor's report on FS' internal control structure over financial reporting identified five material internal control weaknesses, each of which has a direct relation to the program. The auditor's report on Forest Service laws and regulations contains instances of noncompliance with appropriations law and instances of noncompliance with the Federal Financial Management Improvement Act of 1996. The same report also indicated the "USDA Forest Service does not obligate all transactions required by appropriations law" as well as its belief that the USDA Forest Service may not be in compliance with the Anti-Deficiency Act, 31 U.S.C. 1517.

Evidence: FY 2002 (08401-1-FM), FY 2002 and FY 2003 (08401-3-FM), FY 2003 and FY 2004 (08401-4-FM) and FY 2004 and FY 2005 (08401-6-FM) Forest Service Audit Reports from KPMG LLP and reviewed by USDA Office of Inspector General (http://www.usda.gov/oig/rptsauditsfs.htm ) PART guidance states that agency-wide material weaknesses that have a direct relation to the program requires a "No." Guidance further states a program is eligible for a "Yes" only if there are no instances of non-compliance with laws and regulations related to financial management.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Regional program managers develop review teams composed of regional, national and other state participants to conduct program and financial management reviews. The Forest Service's Washington headquarters conducts regional program and general management reviews with every region on a five-year cycle. The agency has dedicated program management staff for oversight and service delivery functions. Regions conduct periodic (semi-annual or annual) meetings with all field program managers to discuss program management issues and policies. An annual national meeting is held to focus on program management issues and policy options to address them. The Forest Service is also consolidating its information technology services based on the result of winning a Competitive Sourcing contract. The Forest Service has also begun implementation of centralized budget, finance and accounting functions for long-term financial management effectiveness.

Evidence: Accomplishment Reporting Requirements; Albuquerque Budget and Finance Service Center; FY 2007 Budget Justification NFVW budget line item narrative on page 8-56.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The Forest Stewardship Program (FSP) provides technical assistance, through State forestry agency partners, to nonindustrial private forest (NIPF) owners to encourage and enable active long-term forest management. Funding is provided to states largely on a formulaic basis, and not awarded either to the states or NIPF owners through a competitive process.

Evidence: USDA Forest Service Strategic Plan for Fiscal Years 2004-08; Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions. See also the Forest Service Manual Sections 1580, 3200 and 3500.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Forest Service provides oversight of the activities of grantee and cooperative agreement participants and tracks their expenditures. Forest Service personnel work closely with cooperators/recipients and clarify regulations, policies, and procedures that may be unclear and/or new to them; determine the need for and, if applicable, conduct post-award site visits or meetings; review financial reports and ensure that payment requests (reimbursables and final requests) are reviewed and approved by the Program Official prior to certification for payment by the Certifying Officer; prepare modifications to the instrument as needed; and determine if corrective action is needed (for example, recommendation for debarment, suspension, financial controls, and so forth).

Evidence: Forest Service Manual Section 1580.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: States, territories, national forests, and other cooperators report annual accomplishments in the Performance Measurement Accountability System (PMAS) through the Forest Service Regions or the Northeastern Area, who then summarize regional or area-wide program and partnership accomplishments for publication in the Forest Service annual accomplishment report. Information is publicly available online.

Evidence: The 2004 accomplishment report is available online (http://www.fs.fed.us/spf/coop/library/2003%20CF%20Accom.pdf). The Forest Service also recently completed a year-long effort to revise cooperative forestry performance measures (http://www.fs.fed.us/spf/coop/library/PMAS%20DeskGuide%202004.pdf).

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: Forest Service Research and Development (R&D) will increase the total R&D funding in extramural work based on merit from the current 13 percent to 20 percent over the next five years. For all other funding and to enhance accountability, R&D managers are required to demonstrate the extent to which their programs meet the following three tests: (1) managers must be able to articulate why this program investment is relevant and high priority; (2) managers must justify how funds will be allocated to ensure quality; and (3) mangers must be able to monitor and document how well this investment is performing. Forest Service R&D has begun an external peer review process to review the relevance, quality, and performance of specific R&D programs, including watershed. This action addresses all R&D investment criteria, but especially focuses on quality.

Evidence: USDA Scientific Research Guidelines and USFS Northeast Research Station Quality Assurance/Quality Control Plan; USFS Science Consistency Review Report and Letter to Stations.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program does not include long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program. The Forest Service will implement a new long-term performance measure begin to track best management practices implementation and effectiveness on all National Forests by FY 2008. The agency is also working to develop a nationally consistent measure of watershed condition.

Evidence: PART guidance provides that a program that received a "No" on Question 2.1 must receive a "No" to Question 4.1.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has had mixed results meeting targets from 2003 to 2005. Watershed management efforts met improvement targets 2 out of 3 years; fisheries met lake enhancement targets 3 out of 3 years and stream targets 1 out of 3 years; wildlife met enhancement targets 1 out of 3 years; wildlife and fish partnership targets were met 3 out of 3 years; range met allotment management targets 3 out of 3 years, but the number of acres of rangeland vegetation improved did not meet goal levels. State and Private Forestry portions of the program and forest management met targets, but only for 1 year each.

Evidence: Forest Service FY 2007 Budget Justification.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Since the program does not include an appropriate efficiency measure are required by PART guidance, it cannot demonstrate improved efficiencies orcost effectiveness. However, some improvement is indirectly evident for some program outputs compared to the base years (fish and wildlife enhancement), and mixed for others (watershed improvements, grazing management).

Evidence: Forest Service FY 2004 - FY 2007 Budget Justifications; FY 2004 - FY 2006 annual program directions. PaRT guidance states a program would normally not be eligible for a "Yes" answer to this question if it received a "No" in Question 3.4.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The performance of this program is on par with other programs managing land and/or conducting similar land-disturbing activities as evidenced by the best management practices effectiveness rating achieved by the Forest Service compared to other Federal agency, State, industry, and private landowners.

Evidence: NCASI Technical Reports 686 (southern states) (http://www.ncasi.org//Publications/Detail.aspx?id=995); 706 (western states) (http://www.ncasi.org//Publications/Detail.aspx?id=1029), and 710 (north central states) (http://www.ncasi.org//Publications/Detail.aspx?id=1041);

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent reviews of Forest Service implementation and of the effectiveness of best management practices (BMPs) applied on National Forest System lands throughout the United States are done on a regular basis by many State water quality agencies. These reviews include both non-Federal and national forest lands, and the Forest Service usually receives high marks. The program has also undergone independent evaluations an independent association of more than 20 fisheries organizations and was indicated to be effective.

Evidence: Rise to the Future 2003 Task Force Report; Programs Assessing Implementation and Effectiveness of State Forest Practice Rules and BMPs in the West by Ice et al., 2004; Forest Practices: Montana's Integrated Approach (http://clfa.org/pdffiles/Montana.pdf); NCASI Technical Reports 686 (southern states) (http://www.ncasi.org//Publications/Detail.aspx?id=995); 706 (western states) (http://www.ncasi.org//Publications/Detail.aspx?id=1029), and 710 (north central states) (http://www.ncasi.org//Publications/Detail.aspx?id=1041);