Detailed Information on the
Federal Lands Highway Program Assessment

Developing revised performance measures in coordination with the National Park Service and implementing a program delivery plan that more clearly links activities with goals and performance.

Develop a comprehensive evaluation plan and conduct a program evaluation in FY 2006.

Measure: Program Delivery Costs (measure/targets adjusted and redefined in FY 2002). This metric measures the percent of funds to deliver projects to construction.

Explanation:Percent of funds to deliver projects to construction. Measure adjusted and redefined in FY 2002.

Measure: Percent of Funds Obligated (measure/targets adjusted and redefined in FY 2002). This metric measures the percent of obligations completed during a fiscal year.

Explanation:Percent of obligations completed during the fiscal year. Measure adjusted and redefined in FY 2002.

Measure: Employee Survey Results

Explanation:Based on a score of 0-100

Measure: Program Administration Customer Satisfaction

Explanation:Based on a score of 0-100

Measure: Project Development Customer Satisfaction. This measure assesses customers' rating of performance by a score of 0 to 100.

Explanation:Based on a score of 0-100

Measure: Completed Project Customer Satisfaction

Explanation:Based on a score of 0-100

Is the program purpose clear?

Explanation: The Federal Lands Highway (FLH) program purpose is to (1) ensure effective and efficient funding and administration for a coordinated program of public roads and bridges serving Federal and Indian lands; (2) to provide needed transportation access for Native Americans; and (3) to protect and enhance our Nation's resources.

Evidence: The Federal Lands Highway program purpose is stated in the Federal Lands Highway Business Plan 2003 - 2007.

Does the program address a specific and existing problem, interest, or need?

Explanation: The FLHP addresses the specific transportation needs, from a national perspective, of the Federal lands which are not a state or local government responsibility. The Federal lands highways (about 160,000 miles of public roads) connect with the National Highway System to provide safe and seamless routes for travel to and within Federal and Indian lands. Federal and Indian lands cover one-third of the Nation's land area. Recent condition surveys of park roads, forest highways, refuge roads, Indian reservation roads, and bridges note recent improvement in conditions, but indicate that thousands of miles of roads and hundreds of bridges need improvement or replacement in order to ensure access to and a coordinated program of public roads. NOTE: Some Federal agencies, such as DOI, transform their pavement condition ratings into asset management systems that also nationally evaluate the status of buildings, construction equipment, and other capital items. States tend to focus on transportation-related items.

Evidence: Appendix E of the FHWA 1999 Conditions and Performance Report provides information on road and bridge conditions (www.fhwa.dot.gov/policy/1999cpr/report.htm). More current information is provided in the Federal agency TEA-21 reauthorization resource papers prepared in fiscal years 2002 and 2003. Other sources of information that discuss program needs are included in the various needs assessment reports and the finalized proposed rule making for management systems.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: As mentioned above, the FLHP provides funding for a coordinated program of public roads that serve the transportation needs of the Federal lands which are not a state or local government responsibility. These highways are critical to the survival and quality of life of tribal communites and other small towns located within these lands. They also connect visitors to the vast number of historic and recreation sites as well as connect tribal housing, schools, health care, and employment centers. The program also fulfills the important role of redistributing Federal transportation funds among Federal Land Management Agencies (FLMA) to ensure mobility and safety throughout 30 percent of the Nation. The Federal program is especially important in 13 western States, some of which have relatively low fiscal capacity and small populations. The FLHP funds are used by the FLMAs for preliminary engineering, design and construction of projects. Maintenance is not a FLHP-eligible item. Maintenance of existing roads and bridges is funded by appropriations directly to these FLMAs.

Evidence: The 2002 Conditions & Performance report indicates that Congress uses this program as the major funding tool to make investments in transportation projects serving Federal and Indian lands. Congress provides separate road and bridge maintenance funding through annual appropriations acts using general funds.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: FLHP funds are distributed to each class of Federal lands highways (forest highways, public land highways, Indian reservation roads, park road and parkways, and refuge roads). Project selection is delegated to the local owners and users (FLMAs, Indian tribes, and States) of the transportation systems, according to three-year transportation inprovement plans (TIPs). The projects included in the TIPs are selected based on relative need. Management systems are also used to identify and prioritize projects. The Public Lands Discretionary program has experienced some eligibility issues and bias in earmarking of various projects. While project selection was once based on competition, the program has now become 100% earmarked. FHWA has addressed this by eliminating this discretionary program through the Administration's reauthorization proposal (the Safe, Accountable, Flexible, and Efficient Transportation Equity Act of 2003, or SAFETEA).

Evidence: Sections 202 and 204 of Title 23 define who selects the projects for each class of Federal lands highways and the criteria to be used. The methods of distributing funds for each class are defined either in section 202 or in policy issued by FHWA or its partners. The requirements to develop three-year transportation improvement plans and management systems are defined in section 204.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The program ensures a connected system of roads that serve local, regional, and national needs by providing resources to each of the land management agencies responsible for 590,000 miles of public roads and highways. Resources effectively reach intended beneficiaries through formula and allocation distribution processes. Several road types owned by the Bureau of Land Management, Forest Service, Bureau of Reclamation, U.S. Corps, and other DOD agencies do not receive dedicated funding and have to compete for funds under a discretionary category. However, this Public Lands Discretionary program is fully earmarked by Congress each year. This issue is being addressed in the SAFETEA reauthorization bill. In the design of the new proposed safety and recreational roads categories, FHWA has proposed the distribution of these program funds to the FLMAs based on performance and results.

Evidence: The beneficiaries of the FLH program are the National Park Service, the Forest Service, the U.S. Fish and Wildlife Service, and the Native American tribes and villages through coordination with the Bureau of Indian Affairs. The SAFETEA proposal would also make the Bureau of Land Management, Bureau of Reclamation, Department of Defense, and U.S. Army Corps of Engineers eligible for percentages of FLH program safety and/or recreational road funding (section 1804 of the SAFETEA bill). Four bills (S. 2884, S. 2906, S. 310,6 and S. 3132) were introduced in the 107th Congress to establish new funding programs to address transportation needs in rural areas of Federal lands.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Federal Lands Highway program is a small and complementary program within the Federal Highway Administration. While FLH activities contribute to the overall FHWA performance goals, the FLH program also has a specific purpose, established by Congress in 1982 (see 1.1). FLH has two long-term performance measures that relate specifically to the first stated program purpose -- to ensure the effective and efficient funding and administration for a coordinated program of public roads and bridges. For its other stated program purposes, FLH contributes to the related FHWA long-term performance measures for mobility and environment. FLH also contributes to additional FHWA and DOT long-term measures, including safety. FLH contributes by collecting and reporting data on condition and performance of road systems, but FLMAs are primarily responsible for the activities associated with these measures.

Evidence: The Federal Lands Highway program purpose and long-term performance measures are stated in the Federal Lands Highway Business Plan 2003 - 2007 (www.fhwa.dot.gov/flh/index.htm); FHWA Performance Plans.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The baselines for the FLH long-term measures have been established through several years of recording data and are ambitious. In addition to working toward achieving FLH's specific targets, FLH also contributes to FHWA's and DOT's long-term targets and "vital few" initiatives as described in the FHWA performance plans.

Evidence: The Federal Lands Highway Business Plan 2003 - 2007 and the 1998 FHWA strategic plan. www.fhwa.dot.gov/flh/bizplan.htm www.fhwa.dot.gov/policy/olsp/strategicplans.htm

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: FLH has annual performance measures that assess the program's progress in achieving both the FLH-specific long-term goals and the FHWA long-term goals, to which FLH contributes indirectly. The measures are based on results from customer surveys. While these outcome measures are acceptable, the addition of output measures (e.g., number of training sessions provided to Native American tribes) could augment the program's annual measures of performance.

Evidence: The Federal Lands Performance Plan 2003 contains the annual performance measures that help to meet long-term goals. The FHWA annual performance plan states annual performance measures to assess progress in achieving long-term goals. Condition and performance measures are described in the FLH Business Plan 2001 - 2005. These have since been turned over to the appropriate FLMAs. The FLH 2003 - 2007 Business Plan has new performance measures that specifically pertain to the FLH program.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: The baselines for the FLH annual measures have been established through several years of recording data and the targets are ambitious. The target of 85% for use in the survey measures was developed in consultation with the national survey consultant group, International Research Institute, Inc. of Fairfax, VA.

Evidence: The Federal Lands Highway Business plans 2001 - 2005, 2003 -2007, and the Federal Lands Highway Performance Plan 2003.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Although they are not required to commit to FLH and FHWA long-term and annual goals, program partners are encouraged to share these goals and FLH has reasonably obtained partner buy-in. Through its joint administration with FLMAs of federally-owned roads, FLH assists partners in accomplishing their missions by providing transportation solutions. Through Memoranda of Agreement (MOAs) with FLMAs, FLH is able to clarify program responsibilities and influence partners to commit to FLH and FHWA goals. In coordinating the FLH program with FLMAs, FHWA stresses the importance of national goals of improving safety, reducing congestion, and protecting the environment. FLH is also sponsoring a performance measurement steering committee to coordinate the transportation performance measures among FLMAs, in an effort to develop common measures. Also, in support of the President's National Park Legacy program, FLH and the National Park Service are coordinating their commitments to further improve the transportation infrastructure within the national parks.

Evidence: FLMAs, States, and tribes report data on number of injuries and fatalities, percent of travel under congested conditions, and the physical condition of roads and bridges. Partners submit information on highway conditions to NHTSA and FHWA for the National Bridge Inventory on a regular basis. NHTSA collects information on highway related injuries and fatalities. MOAs provide written documentation of FLMA responsibilities and commitments. FLMA strategic plans, while not specifically focused on transportation issues, also reference safety and mobility (access) goals.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: While independent evaluations have been conducted for selected portions of the program, there are no comprehensive, regularly scheduled, independent evaluations currently conducted. Both the Departments of the Interior and Transportation Inspector General have evaluated Federal bridges and the Indian Reservation Roads activities. GAO has issued a report on the use of Indian Reservation road administrative funds. Federal Lands has sponsored other independent evaluations in the past, including program reviews by Booze-Allen and peer reviews conducted by the American Society of Civil Engineers. Results from these evaluations have been used to improve program processes.

Evidence: IG and GAO reports: [No. 00-I-597] Independent Auditors Report on Bureau of Indian Affairs Financial Statements for Fiscal Year 1999; [No. 99-i-959] Audit Report on Deferred Maintenance, National Park Service; [No.TR-1998-079] Inspection of Federally Owned Bridges; [No. 96-I-870] Final Audit Report on the Road Construction Program, Bureau of Indian Affairs; GAO/AIMD-00-285R BIA's Use of Highway Trust Fund Resources; GAO/RCED-98-14 TRANSPORTATION INFRASTRUCTURE Review of Project Selection for Five Discretionary Programs; RCED-97-160R Forest Service: Construction of National Forest Roads.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Although FLHP is authorized on a 6-year basis and has not implemented a cost-accounting system, beginning in FY 2004, FHWA has provided a performance-based budget to Congress. FLHP reinforces the linkages between performance and budget through the Administration's reauthorization proposal, SAFETEA. Within this proposal, FLHP has based the allocation of safety and recreational road funding on performance results.

Evidence: FY 2004 budget submission. Also, in support of the safety goal through reauthorization, FLH is proposing to develop and fund a highway safety program for the FLMAs (section 1804 of the proposed SAFETEA legislation).

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: FLH has taken significant steps to address its strategic planning deficiencies, which includes revising adopting long-term and annual goals as well as relevant targets, and measures based on its experience gained as a GPRA pilot. FLH has also revised its strategic planning chapter. With the new business plan these goals and measures have been refined.

Evidence: The Federal Lands Highway Business plans 2001 - 2005 and 2003 -2007; and revised FLH manual chapter on strategic planning.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: FLH collects information on program delivery, program administration, customer satisfaction, injuries, and fatalities, as well as roadway and bridge conditions on a regular basis. While existing authorizing language for the program does not allow much flexibility in using this information to make resource reallocations, the information is used to manage the program in other ways. For example, collected performance information is used during the "August redistribution" process to move funds from agencies whose projects have not met schedules to others that are ready to be awarded. Also, information is used to assign workload for staff and contractors, as was done for mega-projects. Further, performance requirements are proposed in SAFETEA for the new safety and recreational roads programs, for which funds to the FLMAs would be distributed based on performance results.

Evidence: The FHWA Performance Plan and the Conditions and Performance report publish data on performance that is routinely collected from the Federal partners and States (forest highways). Condition information is also reported in Federal agency reauthorization resource papers. Section 1804 of the proposed SAFETEA legislation.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: FLMAs and States (forest highways) are generally held accountable by the nature of the formulas and set amounts for each funding category. FLH requires additional financial accountability and reporting of obligations by partners. FLH program managers are held accountable to their commitments through FHWA's performance appraisal system and those who do not perform are relegated to other support roles. Regional FLMA management holds their local planners accountable for completing their share of the overall FLH program. If projects miss deadlines, they are not constructed and returned to the queue of projects to again compete with other projects from across the country. As per their contract, contractors must meet schedule and performance requirements and are subject to fines for not doing so. Contractors who do not meet their contract requirements and/or are found guilty of waste, fraud, and abuse may lose their license and can be barred from bidding on government jobs.

Evidence: FLH works with FLMAs and States (forest highways) to develop transportation improvement program and projects. FHWA has MOUs with other Federal agencies for most FLH program funding categories and tri-party agreements with States for forest highways. Indian tribes perform activities and projects funded through the Indian reservation road program under P.L. 93-638 contracts and agreements.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The majority of funds for this program are obligated in a timely manner. Public lands highway funding earmarked for projects by Congress is occasionally not obligated in a timely manner because funding is earmarked before projects are ready to begin. Nearly all of the funds for this program are spent for the intended purpose. Some small amounts of Indian Reservation Road funds were mismanaged, but these occurrences are an anomaly.

Evidence: SF 133 reports and FHWA account reports on FHWA, Federal agencies, and State (forest highways and public lands highways) obligations show that almost all funding is obligated in a timely manner.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program's performance plans include efficiency measures and targets. In FY 2000, FHWA identified 35 positions as potentially available for competitive sourcing and successfully converted to contract 15% of those positions in accordance with OMB guidance. In its FY 2002 inventory, the FHWA identified 245 positions that were in commercial functions, of which 14 positions reside within FLH. FHWA and FLH are pursuing a goal of competing between 40 and 50 percent of the FY 2002 inventory positions beginning in FY 2004. In addition to the President's Management Agenda challenge that drives the Competitive Sourcing initiatives, FLH historically contracts a large percent (40%) of its program activities. The requirement to use consultant services has been driven by FTE requirements and not necessarily by cost benefit comparisons.

Evidence: FHWA and FLH Performance Plans contain detailed performance measurements. FHWA also has business cases for its relevant programs. Cost information comparing Federal workforce versus consultant services is available in every FLH division office. Project managers prepare their preliminary engineering design cost estimates based on whether the job is to be designed within the office or by outside contractors. Prices vary between consultants and the date of their indefinite delivery, indefinite quantity (IDIQ) contracts.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Program partners coordinate their maintenance and improvement activities with the FLH program funded projects that they administer. FHWA collaborates and coordinates with its division offices, other Federal agencies, States, local governments, and transportation organizations. FHWA works closely with NHTSA and FMCSA on safety and freight programs and with EPA on strategic planning and environment programs. FHWA also sponsors local meetings and outreach programs.

Evidence: DOT strategic and performance plans.

Does the program use strong financial management practices?

Explanation: FHWA uses effective financial management practices in administering program funds. FHWA is in the process of updating their DAFIS financial management system with a new DELPHI system. FLH, through the FHWA Budget office, is also involved in implementing a new managerial cost accounting program. All of these financial programs have strong error prevention features.

Evidence: FHWA's erroneous payments rate is less than one percent and the program has received clean audits in recent years.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: FHWA has implemented a new financial management system and a protocol for identifying and recovering erroneous payments. FLH also participates in the dashboard performance measurement system initiated by Administrator Peters. This system gives top management a monthly view of the progress and performance of the agency on key focus areas.

Evidence: FHWA has provided a detailed description of its erroneous payments program. The new financial management system tracks comprehensive financial data.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: FHWA has sufficient oversight capacity and receives information from Federal agencies, Indian tribes and States (forest highways and public lands highways) on their Transportation Improvement Programs (TIPs) and planning activities. FLH and partner staff conduct field reviews of selected project and program activities. FLH is also participating in the agency's incorporation of risk management techniques in selecting partner program areas that are prime for review.

Evidence: FHWA has published an oversight policy, and the Conditions & Performance Report and Transportation Improvement Programs (TIPs) provide information on grantee activities.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program collects performance data on an annual or bi-annual basis. Most partners contribute to this data collection. FLH also collects some of this data for its program partners. The FLH program makes its performance data available to the public via its four websites and through its various publications.

Evidence: Partners submit information on highway conditions to FLH and bridge conditions to the National Bridge Inventory on a regular basis. NHTSA collects information on highway related injuries and fatalities. Condition information is found in the Conditions and Performance report and Federal agency resource papers. Web sites: www.wfl.fhwa.dot.gov; www.cflhd.gov; www.efl.fhwa.dot.gov; www.fhwa.dot.gov/flh.

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: While the program has not met all of its long-term performance goals, it has made adequate progress in achieving these outcome goals. The change in results between 2001 and 2002 for measures #1 and #2 is a result of redefinition of costs. The new measures cover all funding that passes through the FLH program including reimbursable funds for work performed for other Federal agencies). The 2001 measure only covered funding allocated to the program.

Evidence: The Federal Lands Highway Business plans 2001 - 2005, 2003 -2007.

Does the program (including program partners) achieve its annual performance goals?

Explanation: While the program has not met all of its annual performance goals, it has shown progress towards meeting targets. Some baselines, targets and measures were recently revised to reflect the experience gained as a GPRA pilot.

Evidence: The Federal Lands Highway Business plans 2001 - 2005, 2003 -2007; draft FLH manual chapter on performance measures; historical data from 1989 to present.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The FLH has demonstrated its improved efficiencies and cost effectiveness. In 2000, FLH completed a benchmarking study to optimize its use of contractor services. This study compared outsourcing costs and benefits for 12 State DOTs and several consulting firms. Based on the model developed from this study, FLH is fine-tuning its organizational structure to improve its program delivery effectiveness.

Evidence: FLH 2000 Benchmarking Study and FLH annual reports.

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: In the course of delivering engineering services to other Federal Land Management Agencies (FLMAs), FLH makes use of in-house staff, consulting firms, and various State highway departments. While production results vary on a project-by-project basis, FLMA customer satisfaction survey results indicate that for a variety of reasons, they prefer to have their projects designed and constructed by FLH in-house processes.

Evidence: Customer satisfaction surveys.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Internal documents such as the FHWA Performance Plan, the Conditions and Performance Report, and Federal agency resource papers provide good analysis but are not independent. IG and GAO reports tend to focus on particular aspects of the program, but they do not provide a comprehensive evaluation of the program in relation to specific performance measures. FLH has in the past requested and received peer reviews conducted by the American Society of Civil Engineers and Booze & Allen. In coordination with their FLMA partners, FLH also conducts bi-annual user surveys. The results of these surveys help FLH focus on specific program improvements. While these external reviews indicate that certain aspects of the program are relatively effective and also help the FLH program effectively improve their performance, they are not specifically focused on systematic program reviews.

Evidence: IG and GAO reports: [No. 00-I-597] Independent Auditors Report on Bureau of Indian Affairs Financial Statements for Fiscal Year 1999; [No. 99-i-959] Audit Report on Deferred Maintenance, National Park Service; [No.TR-1998-079] Inspection of Federally Owned Bridges; [No. 96-I-870] Final Audit Report on the Road Construction Program, Bureau of Indian Affairs; GAO/AIMD-00-285R BIA's Use of Highway Trust Fund Resources; GAO/RCED-98-14 TRANSPORTATION INFRASTRUCTURE Review of Project Selection for Five Discretionary Programs; RCED-97-160R Forest Service: Construction of National Forest Roads. ASCE Peer Review reports. Booze-Allen program evaluation report. User survey reports.