Summary of Roundtable on System Performance Measurement
in Statewide and Metropolitan Transportation Planning

Washington, D.C.
October 7-9, 2003

Prepared for:
U.S. Department of Transportation
Office of the Secretary
Federal Highway Administration
Federal Transit Administration

Prepared by:
Volpe National Transportation Systems Center
Research and Special Programs Administration
U.S. Department of Transportation

1.0 Introduction

The Roundtable on System Performance Measurement in Statewide and Metropolitan Transportation Planning was held October 7-9, 2003. The U.S. Department of Transportation (USDOT) Office of the Secretary, Federal Highway Administration (FHWA), and Federal Transit Administration (FTA) sponsored the roundtable. The roundtable brought together national experts to discuss lessons learned in practical applications of transportation performance measures for systems planning and the role of measures in the decision-making process.

Systems planning is conducted at statewide and metropolitan area levels to plan efficient and effective multimodal transportation networks. Networks can include interconnected facilities for roads, public transit, rail, ports, airports, or non-motorized modes, such as bicycles and walking. Systems planning considers transportation of passengers and goods. The Intermodal Surface Transportation Efficiency Act (ISTEA) and its reauthorization under the Transportation Equity Act for the 21st Century (TEA-21) encourage a coordinated intermodal approach to planning transportation systems.

Roundtable participants included experts from state Departments of Transportation (DOTs), metropolitan planning organizations (MPOs), public transit operators, research centers, consultants, and USDOT. Participants represented the following organizations (listed in Appendix A):

The USDOT organized the roundtable in the following five moderated sessions to provide a structure to exchange experiences and ideas.

1. Overview of the state of the practice
2. Selected examples of effective practice
3. Common trends in applications
4. Requirements for successful use of performance measures
5. Agenda for the future

This report describes a range of examples of applications of performance measures for systems planning at statewide, metropolitan area, and public transit levels. It also identifies national trends in applications and related concerns and concludes with a "strawman" agenda for the future.

This report is intended to be a resource for transportation planners at state DOTs, MPOs, and public transit operating agencies interested in applying performance measures to systems planning.

The report uses the following sources provided by participants:

• Discussion during the roundtable.
• Summaries on performance measure applications by organizations represented (provided in Appendix B).
• References to planning reports of participants' organizations (with web-links).

The participants identified the following aspects as critical to successful application of measures for systems planning:

• Leadership
• New tools and data
• Communications skills
• Incentives

Participants identified a broad range of applications of performance measures for systems planning, including:

1. Strategic Planning: To measure the cost and performance of alternative packages of investments and strategies in long-range visions or scenarios for states or metropolitan areas.
2. Selection of Alternative Investments: To compare, prioritize, and select alternative modal investments or strategies.
3. Monitoring Performance: To monitor the performance of transportation organizations, as well as operations of the multimodal system.
4. Accountability: For accountability within transportation agencies or between agencies and others, particularly elected officials, the public, and stakeholders.
5. Communications: For communication with elected officials, the public, and stakeholders to provide an understanding of the performance of the transportation system, and critical choices involving that system.

2.0 Trends and Issues

This section includes discussion of the following seven trends, and related issues. It also provides examples of successful applications identified by participants, with references to web-sites and planning documents.

1. Technical challenges
2. Institutional issues
3. Communication of system performance
4. Non-traditional measures
5. Moving beyond modal and organizational competition for resources
6. Identifying when to invest
7. Cost of data and successful applications

Most of the examples can be placed in more than one of the above sections, indicating the crosscutting nature of innovative applications of performance measures for systems planning. For example, the same application by Maryland DOT (discussed below) demonstrates the use of measures to establish political and public credibility (communication of system performance) and the combination of mobility, accessibility, and other performance measures at a system level (non-traditional measures).

2.1 Technical Challenges

Participants described how their agencies use performance measures to plan, program investments, and manage operations. Some examples can be considered modal – for example, levels of service, safety rates for highways, farebox recovery and load factor, passenger miles per vehicle mile or hour, or schedule adherence for public transit. These applications tend to be well-established aspects of modal planning. More challenging, however, is how to develop and successfully apply additional measures to reflect the performance of all modes as part of an integrated system; this challenge was a focus of the roundtable.

As mentioned in the roundtable, "what gets measured gets funded," which has important corollaries – what gets measured can be managed, planned, and communicated to decision-makers who may not otherwise think in terms of system performance.

From the discussion it was apparent that there is no single fixed and universal set of measures for systems planning. Identification of measures will depend on the participating organizations' roles and responsibilities as well as goals and priorities, whether as a state DOT, transit agency, or MPO. Different agencies focus on different measures; however, the definition of a system should focus on the identification of measures of the entire system, rather than its modal performance or integration of passenger and goods measures. This will require clear communication and agreement among participating agencies to determine what constitutes the transportation system for a state or region, what goals to plan for, and what performance to monitor.

The following examples are of initiatives of a single organization or group of organizations either focusing initially on systems measures, or expanding from a base that measures modal performance to also plan for system performance.

Ohio DOT's (ODOT) long-range transportation plan, Access Ohio, describes statewide changes and trends in traffic volume, mobility, and population. Figure 1 presents statewide trends in freight transportation from 1998 to 2020 across water, rail, and truck modes using the single measure of tons. The graphic demonstrates both a statewide net increase in freight movement for all modes as well as significant shifts in forecasts from water and rail to trucks.

Figure 1: Ohio Freight Movement by Mode

The Ohio DOT also uses performance measures to analyze transportation system trends in its twelve districts. Figure 2 presents the key performance measure of congested lane miles across the districts, indicating distribution of congestion in the statewide system between districts, with clear identification of three districts with major levels of congestion. This single performance measure has implications both for freight and passengers.

Figure 2: ODOT's Geographic Distribution of Congestion

ODOT plans to expand its use of performance measures in the update to Access Ohio, which will incorporate improved performance measure-based analysis, and forecasting of bridge and pavement conditions, sufficiency analyses, new corridor analyses, freight analysis, congestion analysis, and other system analysis. It will also incorporate the MPO long-range plans and public opinion.

TriMet operates the Portland, Oregon metropolitan area public transit system, including fixed route bus and the MAX (Metropolitan Area Express) light rail system. TriMet uses performance measures to monitor and communicate both organizational and regional performance and agency-wide progress. It accomplishes this while emphasizing the links between transit and regional land use and long-range transportation plans. TriMet produces a Monthly Performance Report for its management summarizing financial and operating performance. The report includes measures of system ridership, revenues, cost efficiency, customer service responses, and maintenance performance. TriMet produces statistics for the current and prior months, the current month in the previous year, and the 12-month averages for the current and prior years. Departments post performance reports to the TriMet website to help in analysis of operations using selected performance measures to describe progress.

For example, TriMet created Figure 3 as part of the MAX System Overview (http://www.trimet.org/inside/history/maxpage.htm#ridership). The graphic shows how key operational measures of transit service -- vehicle miles traveled (VMT) and transit ridership -- can be contrasted to population growth to demonstrate performance to elected officials, policy makers, and the public. The measures place transit performance in a regional and systems context by juxtaposing it against population growth (ridership increased faster), auto use (despite growth in transit service, automobile VMT increased faster), and transit ridership (increased faster than transit service, demonstrating efficiency). From a broader perspective, with land use dimensions, the graphic demonstrates that the growth in travel (auto and transit) greatly exceeds the ten-year growth in population.

Figure 3: Measuring transit performance in relation to population increase and auto increase

2.2 Institutional Issues

Planning agencies are often faced with overcoming institutional barriers before performance measures can be successfully applied to systems planning. Each agency should overcome institutional challenges internally and when collaborating with other agencies. Most agencies need to continuously promote the importance of system performance measures, which may not be fully accepted or appreciated by staff and decision-makers, who may have been trained in or worked with engineering or other well-established modal performance measures. Decision-makers can be encouraged both to consider and invest in development and application of broader sets of measures.

Acceptance of performance measures agency-wide and in collaboration among agencies can contribute to effective applications at the systems level, but this case should continually be made and not assumed. As designated agencies for regional planning, MPOs face expectations to coordinate with the state DOT as well as transit and other modal planning and operating agencies. Typically, MPOs have significant roles as brokers and communicators as metropolitan area transportation planning agencies. This can involve convincing modal partners to assess their contribution to system-wide performance, and to participate with other agencies in identifying and applying system-level measures. Common frameworks of measures, whether at statewide or metropolitan levels, can encourage and make it easier for agencies to coordinate and collaborate technically and institutionally, as required for successful systems planning.

Participants discussed whether "mandates" at federal or other governmental levels could encourage use of performance measures in systems planning. Opinions ranged from support of federal mandates to disagreement that even limited mandates could play a helpful role.

Performance measures help make the decision-making process transparent and contribute to critical accountability within and among organizations and between organizations and elected officials and the public.

Washington State DOT's "Gray Book" explains the agency's planning process and the rationale for why different steps occur. In addition to the Washington State DOT, the Washington State Transportation Commission also assesses the effectiveness of the statewide system. The Commission serves as a geographically and politically balanced agency that applies an institutional approach to measuring performance. Together with the Washington State DOT, the Commission has instituted new congestion measurements to better address above-average commute times. Data from the 2000 Census showed Washington as having the eleventh longest average commute time among the states. To address this concern and others, the Transportation Commission's Benchmark Committee adopted new congestion measurement principles in 2001. The DOT uses these measures not only to assess highway congestion, but also to be used as a means to determine the success of the DOT's tools and actions.

While a reporting tool such as the Gray Book is a consistent method to encourage accountability, the agency believes that immediate reporting tools tied to investment may have some limitations. For example, although rural areas may not experience congestion, a major measure of statewide performance, they may still require investments. By linking a limited aspect of performance to investment, rural areas may receive minimal funding because they are not highly populated areas, but are still in need of improvements and investments perhaps based on safety, economic development, accessibility to critical services or other goals. Without an adequate set of measures the risk is that projects that satisfy other priority statewide goals may be deprived of funding, resulting in a reduced level of system performance.

Washington State DOT views communication as the critical element of successful application of performance measures. The agency publishes a quarterly report on performance measures, Measures, Markers and Mileposts (http://www.wsdot.wa.gov/accountability/graynotebook.pdf). The report provides accountability of the DOT to the State Transportation Commission and public.

Ohio DOT developed the Organizational Performance Index (OPI) to monitor agency performance based on key objectives. This approach focuses on internal and external expectations, and ensures accountability for performance. In Ohio, the DOT has decentralized key responsibilities to its twelve district offices. The measures are used to evaluate performance across the state, within the districts, and by the district offices against statewide averages. The OPI system is refined continually to ensure that accurate metrics are used to measure DOT's performance. With decentralization, the measures encourage consistent results and improvement of performance among the districts. The system includes measures in the following categories:

• Construction management
• Contract administration
• Equipment and facilities
• Finance
• Information technology
• Maintenance
• Plan delivery
• Quality
• Human resources,
• System conditions and
• Traffic

Figure 4 provides examples of Ohio DOT's performance measures for four of these categories. The Measurements of Organizational Performance are produced at: http://www.dot.state.oh.us/dist12/PDF%20files/info%20and%20analysis.pdf.

Figure 4: Ohio DOT's OPI measures for daily operation activities.

The Montana DOT Strategic Business Plan provides an innovative combined application of transportation performance measures and organizational performance measures, linking employees to the "global success" of the agency. The business plan is "a living document" that is the basis for business actions and performance measurement for departments and individuals. The plan is designed to ensure that employees understand the agency's goals, standards, and activities. In addition, employees will have individual performance plans based on the agency's twelve strategic initiatives. The Strategic Business Plan is available at http://www.mdt.mt.gov/publications/docs/brochures/business_plan.pdf.

2.3 Communication of System Performance

It is critical for decision-makers and the public to understand performance measures. While measures should be comprehensive, they also should be clear and concise to engage decision-makers and the public and hold their interest through the planning and decision-making process. Because decision-makers might understandably have a focus on modal or jurisdictional concerns, it is important to communicate the importance of adding a system perspective. By documenting successful applications from other states, MPOs, or transit authorities, decision-makers may appreciate and gain confidence in the value of new systems measures.

Washington State DOT and Florida DOT have developed specific programs to address this concern. Washington State DOT's accountability committee represents the public and legislature, and addresses what measures to use in assessing performance. Florida DOT's report, Government Accountability for the People, communicates agency performance to the public in an uncomplicated fashion.

Maryland DOT uses performance measures to raise its political and public credibility by demonstrating customer satisfaction as one element of successful performance. The 2003 Annual Report included "Serving our Customers" as one goal with specified performance measures. Figure 5 lists related measures and targets for Maryland's Motor Vehicle Administration and Transit Administration.

Figure 5: Maryland Performance Measures and Targets for Customer Satisfaction

Additionally, Maryland DOT provides detailed analysis of the benefits of future investments to gain support for new funding. The 2003 Annual Attainment report demonstrated the need for more funding through a comparison of the DOT's expected capital program and the budget recommendation of the Commission of Transportation Investment. While these are just two examples of how Maryland DOT communicates with the public, the agency continues to strive for performance measures that can be attained and accepted at a bi-partisan level and that clearly communicate with both the public and decision-makers.

The Binghamton Metropolitan Transportation Study (BMTS), the Binghamton, New York area MPO, communicates its goals and performance objectives in its long-range plan, Transportation Tomorrow: 2025. To maintain the credibility and validity of the Plan, BMTS reviews performance annually as well as in detail in the five-year cycle reevaluation. By including specific goals and objectives for mobility, safety, and level of service, BMTS can be held accountable for performance based on related activities. The goals, which reflect investment priorities and have associated performance measures for multiple modes, include:

1. System Preservation and Maintenance: Reduce number of deficient bridges by 10% to improve and maintain pavement efficiency; replace buses operating beyond FTA-designated lifetimes.
2. Personal Mobility: Designate investments to improve all modal travel and overall customer experience. Investments include high priority and other metropolitan corridors, and improvements to automobile, public transit, shared ride, and other modes.
3. System Management and Operations: To achieve an effective management and operation of the metropolitan transportation system, develop partnerships with transportation organizations and further opportunities to implement ITS to improve operations.
4. Safety: Adopt a proactive approach to safety throughout all stages of project development and implementation, focusing on capital projects, public transit facilities, high accident areas, and additional modal concerns.
5. Economic Growth and Freight Mobility: Identify a priority truck route system within one year of long-range plan implementation and support proposal for truck-rail intermodal terminal.
6. Environmental Protection and Quality of Life: Encourage communities to implement traffic calming and enhancement funds for bicycle and pedestrian facilities.

In particular, the last three goals communicate performance aspects of broad political and public interest. The annual performance review of the plan is available at http://www.gobroomecounty.com/bmts/pdfs/BMTSApr05Minutes.pdf.

Washington State DOT has developed the following series of Congestion Measurement Principles to communicate critical technical dimensions in a meaningful way to non-technical stakeholders.

• Use real time measurements (rather than computer models) whenever possible.
• Measure congestion due to incidents as distinct from congestion due to inadequate capacity.
• Show whether reducing congestion from incidents will improve travel time reliability.
• Demonstrate both long-term trends and short-to-intermediate term results.
• Communicate about possible congestion fixes using an "apples-to-apples" comparison with the current situation (for example, if the trip takes 20 minutes today, how many minutes shorter will it be if we improve the interchanges?)
• Use plain English to describe measurements.

2.4 Role of "Non-Traditional" Measures

Agencies can include non-traditional measures to capture additional aspects of performance in terms of broad public policies. These aspects can be in addition to and complementary to more traditional measures of transportation performance, such as highway levels of service, average speed, or mobility. Non-traditional measures can be useful when communicating system performance broadly, beyond those with a specific focus on transportation. For example, energy and environmental agencies and their stakeholders will be particularly interested in energy use and efficiency and environmental protection of the system.

Combining measures from other sectors, such as land use or economic development, with transportation measures can also expand how transportation system performance is measured. Land use agencies use technologies such as geographic information systems to identify percentages of population groups served by transit or major arterials within time and distance radii and development around transportation infrastructure. These combined measures can be very helpful in equity analysis by contrasting access to transportation provided to different income or demographic groups.

Agencies attempt to apply other non-traditional measures to assess transportation impact on quality of life or sustainability. The latter seeks to balance equity, economy, and environmental concerns -- issues that often arise during the environmental review process. Several cities have offices with specific responsibilities in these areas, and are concerned with important aspects of transportation planning.

Roundtable participants suggested the need to consider transportation measures that reflect performance related to environmental, quality of life, or other issues as part of the systems planning process. It is often challenging for measures to address aspects such as the quality of life or "livability" and may require consideration of new performance measures.

Minnesota DOT has compliance measures in place to account for land use and air and water quality, while the Southern California Association of Governments (SCAG) addresses air quality as a critical transportation system performance measure in response to Federal requirements under the Clean Air Act Amendments as well as TEA-21.

Maryland DOT's 2004 Annual Attainment Report on Transportation System Performance evaluates the performance of the Maryland DOT, Maryland Transit Authority, and the five modal administrations, and how these agencies have met the goals outlined in the Maryland Transportation Plan (MTP).

The report includes 30 performance measures and emphasizes the following key goals, with examples of associated measures:

• Efficiency: Acceptable ride quality, structurally sound bridges, and on-time transit service.
• Mobility: Congestion levels and vehicle trips served by E-Z Pass.
• Safety and Security: Number and rate of injury accidents and fatalities, customer perspective.
• Productivity and Quality: Transportation-related emissions by region, transit operating costs per passenger, maintenance expenditures per mile, and customer feedback.

Innovative aspects include: combination of road and transit measures to reflect efficiency; including E-Z Pass trips along with the core measures of congestion levels; combination of qualitative customer perspectives with quantitative injuries and fatalities; and for the broad category of productivity and quality, combining emissions, costs per passenger, and maintenance expenditures alongside customer feedback.

The 2004 Report is available at: http://www.mdot.state.md.us/bin/m/f/Final%202004%20Attainment%20Report.pdf

TriMet's regional performance measures are described in the 2000 Regional Transportation Plan (RTP), which expands the use of congestion as a measure to include in regional policy. Areas designated as "special concerns" can apply a broader definition of performance in mixed-use centers and corridors, where transportation solutions solely aimed at relieving congestion are inappropriate for functional, physical, financial or environmental reasons. Future RTP updates will also include the expanded definition and applicability of performance measures to all modes.

TriMet's special attention to environmental factors also plays a key role in assessing their performance. TriMet recognizes the need to improve air quality and reduce emissions, and uses the following measure to demonstrate performance in improving air quality and congestion:

• TriMet's services reduce smog-causing pollutants by nearly 4.2 tons every day.
• On average, Portland-area residents make 187,000 fewer daily car trips due to TriMet's services.

Metro, the Portland area MPO, first published the "Performance Measures Report" in March 2003 to evaluate the effectiveness of the region's growth management policies. As Metro's first-ever report on the progress being made toward meeting the goals of the region's 50-year growth plan, the 2040 Growth Concept, this 19-page performance measure report is a snapshot of how the Portland region is doing in relation to Metro's growth management goals. Not only does the report use performance indicators to address development of a balanced transportation system, it also employs measures to consider that system within the context of values for land use and transportation performance that include:

• Encourage a strong local economy
• Encourage efficient land use
• Protect and restore the natural environment
• Maintain separation between the Metro urban growth boundary and neighboring cities
• Create a vibrant place to live and work
• Provide a balanced transportation system

To demonstrate the value of a balanced transportation system, the report evaluates congestion by examining changes in freeway volumes on major corridors, area-wide total VMT and per capita VMT, and transit ridership from 1990 to 2000. The report also considers average annual capital needs relative to capital spending for roads, transit, and non-motorized modes. The report is available at: http://www.metro-region.org/library_docs/land_use/2040_report.pdf

2.5 Moving Beyond Modal and Organizational Competition for Resources

Statewide and metropolitan transportation planning provides an organized and technical means to program fixed and limited resources flexibly and fairly among modes, organizations, and jurisdictions. Flexibility and analysis of costs and benefits of alternative investments is an important aspect of the technical and political decision-making process. Federal, state, or local funds are programmed for projects that can demonstrate the best performance relative to identified goals and priorities. Planners and officials frequently mention establishment and encouragement of a "level playing field" for modes as a strength of the TEA-21 framework and of statewide and metropolitan area planning. Flexible planning and fair comparisons of alternatives relies on clear measures directly related to performance.

When performance measures focus on system-wide performance using agreed upon measures such as reduced congestion or delay, improved accessibility or mobility, safety, or broader measures of environmental protection of "livability," this can encourage improved performance in these areas. The challenge is that the state, city or county, and operating agencies will need to collaborate to agree on systems measures, to collect credible data, and to agree how to apply that data, for example, in prioritization of multimodal projects. Since system performance is by definition multimodal and relies on efficient intermodal connections, success will require partnerships. It will be important to provide incentives for partnerships and other activities that further system performance by agencies accustomed to pursuing different missions and goals. A public transit agency with a focus on farebox recovery and a state DOT division with a focus on congestion and pavement management may also need to find ways to collaborate on other goals, for example, environmental protection or safety.

Ohio DOT provides an example of how state DOTs and MPOs can use performance measures to focus attention on critical aspects of performance both statewide and within the metropolitan areas. The DOT worked with the state's sixteen MPOs to develop one consistent and broadly accepted measure of congestion and is working with the MPOs to expand discussions of consolidated measures of congestion that are consistent with the state's use of measures of mobility to evaluate capital improvements. The Ten-Year Mobility Report identifies universally accepted measures of institutional performance. An overview of the Ten-Year Plan, with performance measures is at http://www.dot.state.oh.us/divplan/pdfs_files/sos2003b.ppt.

Roundtable participants identified this topic as complex, both technically and institutionally. It may be a useful area for emphasis in future research and in development of applications.

2.6 Use of Performance Measures to Identify When to Invest

Performance measures can be used as tools to prioritize projects and strategies in terms of relative success improving system performance. Performance measures can be used to track and project the performance of the existing transportation system. These measures can then be compared with the performance projected to occur under different investment strategies. This can help agencies to assess both weak and strong projects, while determining how funding is being used and where it is needed. Participants observed that identification of weak projects and justification for delaying or dropping them from the programmed list could be very helpful. This is in addition to the more typical focus on prioritizing the most promising projects in an objective and transparent manner that is perceived as fair by competing organizations.

Montana DOT has implemented performance-based planning and programming to ensure that projects receive the proper amount of funding. The DOT allocates roughly 70% of its capital program through the Performance Programming Process, "P3." According to Performance Programming Process: A Tool for Making Decisions, P3 builds support for investments by demonstrating likely performance declines if funding is lost. P3 provides accountability with "feedback loops to monitor predicted versus actual performance." Key aspects of the process are:

1. Being Customer-Driven
2. Having Incremental Development
3. Maintaining a High-Level of Accountability
4. Supporting Sound Investments
5. Maintaining Cross-Cutting Activities

As part of P3, the DOT has established objectives, performance measures, and performance targets in the following program areas:

1. Pavement
   
   Objective: Preserve pavement condition at existing or higher levels.
   
   Performance Measure: Ride Index – quality of ride perceived by highway user.
   
   Performance Target: Interstate – average ride desirable or superior; less than 10 percent of miles below desirable.
    
2. Bridge
   
   Objective: Improve condition of bridges on state highway system.
   
   Performance Measure: Number of functionally obsolete, structurally deficient, and substandard bridges as measured by National Bridge Inventory Condition Assessment.
   
   Performance Target: Reduce number of obsolete, structurally deficient, and substandard bridges on state highway system.
    
3. Safety
   
   Objective: Improve safety of state highway system.
   
   Performance Measure: Number of correctable crash sites funded for improvement.
   
   Performance Target: Reduce number of sites with correctable crash features.
    
4. Congestion
   
   Objective: Maintain and improve congestion levels on rural portion of highway system and improve major interchanges and system operation within urban areas.
   
   Performance Measure: Congestion Index on the highway system – a measure of travel delay
   
   Performance Target: Interstate – Congestion Index better than or equal to Level of Service B.

SCAG has utilized performance measures to assist the elected board in making choices on investments both at the corridor and system level for its past three transportation plans. SCAG's performance measures capture the relationship between transportation and diverse public policy concerns, and inform elected officials and policy makers of the broad array of choices -- and implications -- for investing public and private transportation funds. This performance-based approach to planning provides a more inclusive process that allows input from a wide range of stakeholders, helps foster decisions that reflect a better understanding of the issues, and leads to more cost-effective investments. SCAG has also introduced a focus on financial realism and critical choices for political leaders and advocacy groups to move beyond project "wish lists." SCAG's State of the Region 2002 reports on their recent progress and can be found at http://www.scag.ca.gov/publications/index.htm.

SCAG is also working on a Growth Vision as part of the Compass Project to devise long-range strategies for accommodating the population growth projected for the six counties of the metropolitan area. Aspects include encouraging transportation and land use investments that are mutually supportive, locating new housing and existing jobs and new jobs near existing housing, encouraging transit-oriented development, and promoting a variety of travel choices.

The report, Growth Vision, Preliminary Summary, provides system level performance measures to present graphically the differences between current, 2030 baseline, and a 2030 growth vision scenario in terms of:

• Daily Transit Boardings
• Daily Hours of Delay
• Vehicle Miles Traveled per Person
• Total Vehicle Hours of Travel

2.7 Cost of Data and Successful Applications of Measures

Collecting the data necessary for measures to be applied in systems planning can be costly both in financial and human resources. It is critical that data are of good quality (accurate, timely, and credible) and that good estimation procedures be followed; improvements in either data quality or estimation will support the other. Many agencies lack the time, expertise, and funding to collect and successfully apply this data. It was noted, however, that it is also possible to accomplish important analysis with existing data or by redirecting and improving existing data collection programs without significant cost increases.

Financial Aspects
Successful applications of performance measures for systems planning can be complex and costly. There will be significant costs for developing appropriate indicators that are relevant to systems planning and agreed upon by partner agencies. There will then be additional costs to collect the data, whether through surveys or the use of information technology, and then to apply the data. In times of scarce resources, it is difficult to build a case to boards, legislators, policy makers, and the public that data and analysis can compete for limited transportation investments, alongside direct improvements to facilities and service.

It is equally critical but challenging to convince decision-makers that there is a positive return on investment in data and analysis, especially when analysis may slow the process. Advocates may have to explain how improved analysis will ultimately improve performance. Several participants described how performance measures can help direct resources toward investments that improve performance, and away from projects that will be less successful. Nevertheless, it can be difficult to demonstrate a direct "return on investment" in developing and applying performance measures.

The Montana DOT representative described the planning and programming process (see above) as allowing the agency to move beyond "wish lists" of projects to more realistic analysis of trade-offs among alternative investments in terms of performance and goals.

According to the TriMet participant, that agency used performance measures to support the "Productivity Improvement Program" (PIP), an in-house performance review that after initiation in 1999 is now producing annual savings of $16.8 million. This continuing savings is "without any loss to service nor workforce aggravation," and the "effort has been embraced by the workforce and has raised morale by giving the workforce a direct role in management practices." According to the TriMet participant, this progress would not have been possible without the credibility afforded by performance measures, which bolstered external credibility, and assisted in securing authorization from the State Legislature to increase the regional payroll tax for new service.

Human Resources
Participants observed that the successful development and application of measures, as discussed in this report, requires a broad range of technical and other skills. Planners should not only be skilled technicians, but also be able to understand, collect, and apply data, geographic information systems, or forecasting modes. Planners should also be skilled communicators, able to express complex performance dimensions in the understandable, credible, and significant manner demonstrated in many of the examples in this summary.

Successful applications of the measures discussed in this report may also require human relations skills, considering the complex organizational relationships these applications may entail and the range of interests and skills of individuals involved. Entry-level engineers and planners often do not possess this range of skills, and organizations may not be set-up to provide relevant on-the-job training. Another consideration is that organizations may not always recognize the importance of communications and organizational skills as essential elements for effective performance measurement.

3.0 Progress and Next Steps

Many agencies may not be expanding their use of performance measures for systems planning because of reluctance to implement a "data-heavy" system. However, the application of measures should be seen as evolutionary. These concepts can usually begin as modest ones and expand in breadth and quality over time. Because there is competition for funds to invest in development and application of measures, it is critical to begin to communicate the value of related activities to decision-makers.

Agencies can accept the value of performance measures and systems planning as a means to move from the limits of "silo" approaches that focus on performance of modes or within smaller jurisdictions, toward systems planning and improvements to the overall system. These approaches will improve an agency's accountability, which may be perceived as an immediate benefit to elected officials and the public and be critical to secure stable future support.

Several of the participants described how they had developed their extensive systems of performance measures. The systems used in the Portland area, by TriMet and Metro, evolved over-time, using performance measures to demonstrate links between land use, the regional transportation system, and public transit. Portland demonstrates the importance of progressing beyond measures used by individual organizations to overlapping measures at statewide, regional, and public transit levels. The Caltrans representative described the long-term effort required to develop an intermodal management system with performance measures that are accepted and used at statewide and local levels in the state's complex setting with 18 MPOs and major seaports. The Minnesota DOT representative described the DOT's process of beginning with a shared statewide framework and agreed-upon set of issues, which then is followed by development of common performance measures.

Participants observed that the greatest short-term return from improved use of performance measures for systems planning might be from encouraging efforts by agencies with little experience in these applications. Agencies that are already pursuing these activities will have less difficulty expanding applications; this would provide valuable peer examples. The discussion of future priorities focused more on incremental practical improvement than on advancing the state-of-the-art.

After emphasizing effective practices, common experiences, and requirements for successful applications of performance measures during the first four sessions, the roundtable concluded with a session on "an agenda for the future."

Summarizing insights from the prior sessions, participants developed a "straw-man" agenda to improve the application of performance measures for systems planning.

The group identified three general next steps including:

The USDOT should provide national leadership in demonstrating how performance measures can be incorporated into transportation systems planning.

• The USDOT can disseminate information on how to use performance measures in transportation systems planning through outlets such as the Transportation Research Board, Institute of Transportation Engineers, or Intelligent Transportation Systems (ITS) America. Recent research on performance measures includes that of the Transportation Research Board (TRB) Committee on Performance Measurement and the following reports of the TRB National Cooperative Highway Research Program (NCHRP) and Transit Cooperative Research Program (TCRP):
  - A Guidebook for Performance-Based Transportation Planning (NCHRP Report 446)
  - Performance Measures and Targets for Transportation Asset Management (NCHRP future report)
  - A Guidebook for Developing a Transit Performance-Measurement System (TCRP Report 88: 2003)
  - Performance Measures for State Highway and Transportation Agencies (NCHRP Report 20-24(06)A)

The USDOT, in cooperation with transportation planning organizations, states, and transit agencies, should assist and encourage small or new agencies to implement performance measures in their transportation planning efforts.

• The USDOT can encourage national progress in the development and implementation of transportation system performance measures in metropolitan and statewide transportation planning through a "non-prescriptive approach" involving capacity building, developing tools, and documenting examples of effective practices.

The USDOT should organize statewide or regional meetings of state DOTs, MPOs, and public transit agencies to help them focus on performance measures and systems planning.

• The USDOT can convene regional and statewide stakeholders to discuss, coordinate, and cooperate in the development and application of performance measures in transportation systems planning. Technical and institutional collaboration among agencies is critical for transportation systems planning, yet roundtable participants observed that this often does not happen. Agencies have their own priorities and other factors limit opportunities to focus on improving system performance through the use of performance measures. Roundtable participants noted that it can be helpful to involve other stakeholders when developing non-traditional performance measures, including those related to environmental protection, health, or livability goals.

The USDOT, in cooperation with transportation planning organizations, states, and transit agencies, should undertake the following research and development activities to support the implementation of performance measures in system transportation planning.

1. Create a common framework for states and MPOs to include performance measures in their transportation systems planning.
2. Develop a manual for a standard to measure travel reliability.
3. Develop outreach techniques that share the benefits of performance measure applications.
4. Develop a manual and/or training course on how to communicate performance measures to decisionmakers and the public.
5. Address performance measures on safety.
6. Research overall data requirements to support performance measures over time and for different geographic levels.
7. Determine the economic benefits of operational strategies focusing on the benefits of improved travel time.

In addition, participants identified short and long-term goals and actions to encourage successful application of performance measures nationwide. To improve the use of performance measures in transportation systems planning, the USDOT, in cooperation with associations of transportation planning organizations, the Transportation Research Board and other organizations, should:

In the short term

1. Identify opportunities to bring all stakeholders in the transportation planning process together to develop and implement performance measures.
2. Use a systematic approach such as the USDOT capacity building program to reach transportation agencies not yet using performance measures.
3. Issue best practices on use of performance measures in multimodal transportation systems.
4. Define (1) non-traditional data collection techniques and (2) system software requirements to support performance-based planning, reporting, and management of the transportation system.

And, in the long term

1. Create peer workshops for MPOs at the multi-state level.
2. Develop non-traditional means to collect and share data and performance measures.
3. Develop cost-estimation techniques for "fully-loaded" projects (complete capital and operating costs).
4. Determine the impacts of disinvestments on the performance of the transportation system.
5. Create a methodology or mechanism to estimate travel delays due to work zones.

Appendix A
Roundtable Participants

Appendix B
Descriptions of Applications of Performance Measures

California Department of Transportation

The California Department of Transportation initially began formalizing work in performance measures as a result of the requirements of ISTEA for the six management systems. The requirement for an intermodal management system was a primary factor in pushing forward the Department's discussion, development and use of performance measures in its plans and programs. The requirement in ISTEA for each State to prepare a State Transportation Plan was an additional interrelated factor. California used the then required intermodal management system as a focal point for both internal discussions and discussions with metropolitan planning agencies on performance measures. The Department chose to develop a high-end intermodal transportation management system (ITMS) that is GIS based and has embedded performance measures to evaluate alternative transportation investments. The five categories of performance measures in the ITMS are mobility, environmental, safety, financial, and economic with sub-measures in each category. The California Transportation Plan (approved in 1993) recommended that a separate report on transportation system performance measures be prepared with the goal to develop better transportation system information and ultimately result in better investments in the transportation system statewide. This separate report was completed in 1998 and is the foundation for the Department's continuing work in performance measures. This effort included input from policy makers at every jurisdictional level, a technical advisory committee, and public forums to get local agency and public involvement. A statewide performance measurement conference was held in 1997 as part of the effort to solicit input and advice from national and State leaders, academia and stakeholder agencies.

The Department's current activities in performance measures are focused on refining the measures and indictors from the 1998 report and inculcating performance measurement into policy, plans, and program areas in coordination with regional agencies and local partners. Intensive efforts in this area are being done in our Traffic Operations Division, which has developed a performance measurement framework as part of a plan required by the California Legislature on Transportation Management Systems (TMS). The TMS Master Plan (now in draft) lays out an action plan to improve three main TMS processes: Incident Management, Traffic Control (ramp metering and arterial signal management), and Traveler Information. Overall the goal is to improve transportation performance by reducing congestion and delay. The TMS efforts are pushing the Department's thinking in performance measurements forward and into new areas. Performance measurement is also included in the Department's 2003 draft California Transportation Plan. A separate unit has been created in our Division of Transportation System Information solely for the purpose of advancing performance measurement in the Department and with our transportation partners.

Intensive efforts are being made to bring performance measurement into the Department's State Highway Operations and Protection Program (SHOPP). This program is the transportation system preservation and operations program and includes rehabilitation, safety, mobility and other fundamental projects to preserve and improve the existing (built) system. The Department's commitment to move forward with performance measures is expressed in the Department's Vision and Mission Statement. The Statement itself is "Caltrans Improves Mobility Across California". The statement however includes five goals built around performance measures. The goals are: 1) Safety – achieve the best safety record in the nation, 2) reliability – reduce traveler delays due to roadwork and incidents, 3) performance – deliver record levels of transportation system improvements, 4) flexibility – make transit a more practical travel option, and 5) productivity – improve the efficiency of the transportation system.

At the regional level performance measures are required in all Regional Transportation Plans prepared by both the metropolitan transportation planning agencies and the rural agencies. The plans are the basis for the California Transportation Commission approving the biennial funding programs for regional improvement program funds in California. These documents, referred to as Regional Transportation Improvement Programs (RTIPs), are reviewed by the Commission in relation to the stated performance goals and measures in the Plans.

Cambridge Systematics

Monitoring and evaluating the performance of transportation systems have become important aspects of doing business at the national, State, and metropolitan and local levels. Cambridge Systematics has worked with agencies at all levels of government to develop and implement performance measurement systems. At the national level, CS is assisting FHWA establish a mobility monitoring program for the major urban areas throughout the country and has provided support in developing analytic tools that are the foundation for the biennial Conditions and Performance Report.

At the State level, CS has provided performance-based planning and programming assistance to a wide range of States including Arizona, California, Colorado, Florida, Georgia, Maryland, Michigan, Montana, Oregon, Texas, and Washington. CS has also worked with a number of States on developing program delivery and organizational effectiveness measures as well as conducting performance audits of various functions or activities. At the metropolitan and local levels, in addition to our work on the national mobility monitoring program, CS has worked directly with agencies attempting to establish performance measurement systems including Metro Council in the Twin Cities, Santa Clara County and the East-West Gateway Coordinating Council in St. Louis.

CS has also completed one NCHRP project and is currently engaged in a second that focuses on performance measurement:

NCHRP Report 446 – A Guidebook for Performance-Based Transportation Planning. Cambridge Systematics developed a guidebook for performance-based transportation planning that identifies the basic principles of performance measurement, defines a process for developing and implementing performance-based planning, and discusses the data and tools required to support the use of performance measures in planning. An appendix to the guide includes a library of multimodal performance measures that cover freight as well as passenger transportation.

NCHRP 20-60 Performance Measures and Targets for Transportation Asset Management - Cambridge Systematics has just started a new project focused on developing practical guidance for implementing performance measures that strengthen asset management efforts. The objectives of this project are to (1) recommend performance measures that can improve resource allocation in the areas of facility preservation, operations, and improvement; and (2) develop a framework to help transportation agencies select suitable performance measures and set performance targets.

Cambridge Systematics, Inc. provides management and planning consulting services and information systems to the public and private transportation industries. As an independent, employee-owned company, we deliver need-based solutions that help our clients solve current and future problems. Our headquarters is in Cambridge, Massachusetts, with offices in Oakland, California; Washington, D.C.; Chicago, Illinois; and Tallahassee, Florida. Cambridge Systematics serves a variety of local, State, national, and international agencies, as well as transportation, logistics, telecommunications, and manufacturing companies; financial institutions; and other private corporations and business organizations.

Central Ohio Transit Authority

The Central Ohio Transit Authority (COTA) provides transportation solutions to the Central Ohio community primarily by means of bus services. In order to effectively evaluate COTA's transportation system, we have developed "new route" and "schedule design" standards along with route performance evaluation measures based on best practices in the industry. COTA provides primarily four types of services:

Service Categories

• Local (Fixed) routes make all stops and operate between downtown Columbus and various neighborhoods or townships within Franklin County. The vast majority of COTA vehicle service hours are operated on local routes.
• Express routes operate to provide fast, line-haul service between downtown Columbus and suburban areas and/or park and ride facilities. Service is usually operated only in the peak periods (AM and PM) in the peak direction of travel. However, some routes serve "reverse commute" markets and operate in the non-peak direction of travel (i.e., from downtown to outlying employers in the AM).
• Crosstown routes operate between various neighborhoods or townships within Franklin County and do not serve the downtown area.
• COTA operates LINK routes in downtown Columbus and in suburban activity centers. LINK routes are generally short routes that are intended to serve short, non-work trips or serve as feeders to local and/or express routes.

Key Performance Measurements: Productivity and Economic Standards

COTA uses ridership productivity and economic subsidy as its primary measures of transit performance. Ridership productivity is measured in terms of riders per revenue hour of service (Local, Crosstown and LINK routes) or riders per revenue trip (Express routes). Economic performance is measured by calculating the subsidy per passenger trip (boarding). Routes within each service category are ranked according to the productivity measures and compared to the minimum standards identified for each service category. A productivity rating is then calculated for each route and a corresponding list of actions (e.g., marketing promotions, service modifications, elimination, etc.) are identified for further evaluation.

New routes should meet the applicable standards for the service category after one year of operation. All routes are reviewed each trimester and routes that have not shown adequate progress toward meeting the standards are targeted for marketing promotions or possible service modifications to increase productivity.

Florida Department of Transportation

Implementation of the 2020 Florida Transportation Plan – Linkage with the 2020 FTP and the Department's Strategic Objectives

Florida's Transportation Mission – Florida will provide and manage a safe transportation system that ensures the mobility of people and goods, while enhancing economic competitiveness and the quality of our environment and communities.

Goals in the 2020 Florida Transportation Plan

Goal 1 – Safety	Goal 2 – Preservation
Goal 3 - Economic Competitiveness	Goal 4 - Quality of Life

Department Mission - The Department will provide a safe transportation system that ensures the mobility of people and goods, enhances economic prosperity and preserves the quality of our environment and communities.

Strategic Goal – Preserve and Manage an Efficient Transportation System

Focus Area – System Preservation

• Through 2011, ensure that 80 percent of pavement on the State Highway System meets Department standards.
• Through 2011, ensure that 90 percent of FDOT-maintained bridges meet Department standards while keeping all FDOT-maintained bridges open to the public safe.
• Through 2011, achieve 100 percent of the acceptable maintenance standard on the State Highway System.

Focus Area – System Efficiency

• By 2011, improve system efficiency by deploying Intelligent Transportation System Technology on critical state corridors.
• By 2011, improve safety and traffic flow by reducing the number of commercial vehicle crashes on the State Highway System to or below 7.7 per 100 million vehicle miles traveled.

Strategic Goal - Enhance Florida's Economic Competitiveness and Quality of Life

Focus Area – Mobility/Economic Competitiveness

• Through 2007, at a minimum, maintain the rate of change in person hours of delay on the Florida Intrastate Highway System (FIHS).
• Through 2011, commit approximately 50 percent of the highway capacity improvement program for capacity improvements on the FIHS.
• Through 2011, increase transit ridership at twice the average rate of population growth.
• Short Range Objectives relating to economic competitiveness and mobility in non-urbanized areas to be developed.

Focus Area – Safety

• By 2006, reduce the highway fatality rate on all public roads to or below 1.75 fatalities per 100 million vehicle miles traveled.
• By 2006, reduce the fatality rate on the State Highway System to or below 1.54 fatalities per 100 million vehicle miles traveled.
• By 2011, reduce the bicycle fatality rate to or below 0.19 fatalities per 100,000 population.
• By 2011, reduce the pedestrian fatality rate to or below 2.35 fatalities per 100,000 population.

Focus Area – Quality of Life

• Ensure that transportation investment decisions consider the impact on citizens, communities, cultural resources, and the environment.

Draft: September 23, 2004

Maryland Department of Transportation

The Maryland Department of Transportation uses several performance measure systems to help manage assets, identify policies or programs that need additional emphasis, highlight areas not being adequately addressed and recognize successes. These systems range from macro systems levels measures to micro operational measures. Two of the programs we use to respond to legislative mandates for performance measures are the Annual Attainment Report on Transportation System Performance and the Managing for Results Initiative (MFR).

The Annual Attainment Report on Transportation System Performance is a result of legislation passed during the 2000 state legislative session. This legislation revised the requirements for the statewide long range transportation plan (Maryland Transportation Plan), created a requirement for an Annual Attainment Report on Transportation System Performance, and created a Governor's Advisory Committee to assist in the development of performance measures. This Attainment Report is intended to provide insight into the Department's effectiveness in achieving longer-term policy goals by tying performance measurement to both the long-range plan and capital program. The Report is required to include measures to quantify the goals and objectives of the long-range plan and capital program and set targets or use benchmarks to judge these measures. Specific measures that must be included are: increase in share of non-SOV modes, decrease in traffic congestion indicators, cost effectiveness of investments for congestion and cost per passenger mile. The initial emphasis has been on the development of performance measures and targets. Now that these are fairly well established, additional focus is being devoted to the linkage with the budgetary process and cost effectiveness.

The MFR initiative measures shorter-term operation facets of the Department. The MFR is tied into the budget, and measures are submitted with our budget each year. Information from these measures (that reflect the Department's strategic goals) is examined by the state legislature when looking at program funding and when allocating resources based on successful achievement of performance targets. This is a relatively new program and is still evolving. To date, the main link of MFR performance measures to the budget has focused on requests for additional programs funding, more so than any reductions in program funding.

Keeping the measures meaningful, manageable, measurable, and achievable is one the biggest challenges for MDOT. Currently the Department has 45 measures in the Attainment Report (over 100 measures were recommended during development) and close to 100 measures in the MFR document. The Department is constantly trying to strike a balance between having measures to cover all facets of the strategic plan and keeping the number and type of measures in a form useful to the public and at that reflects the appropriate level of measurement (system/policy v. operational).

For more information, please visit: http://www.e-mdot.com/Planning/Plans%20Programs%20Reports/Reports/Attainment%20Reports/2003MarylandAttainmentReport.pdf

Metropolitan Transportation Commission (MTC)- San Francisco Bay Area

Transportation System Monitoring
MTC and Caltrans District 4 jointly publish an annual system monitoring report on transportation system performance. The State of the System report includes data collected by a number of partner transportation agencies and covers performance of freeways, local roads, transit and goods movement. The report highlights customer-oriented measures of mobility, safety and the state of repair. Topics that typically garner the most interest among the press and the public include freeway congestion, travel time to major cities and local road pavement condition. While the report relies principally on existing data, MTC is pursuing collection of some additional data to fill some gaps in performance information.

Long Range Planning
In the fall of 2003, MTC will undertake evaluation of projects under consideration for the Bay Area's long range transportation plan. Legislation enacted in 2002 requires MTC to evaluate all individual projects and investment programs that are not in the current long range plan. MTC is inviting members of the public as well as agencies to submit projects for consideration. MTC developed a two-part assessment in order to address the comprehensive analysis that is required:

• The Project Needs Assessment will consider the underlying needs projects are intended to address. This analysis will rely principally on forecasts from the regional travel demand model to assess transportation system conditions in the future. Some measures are qualitative in nature, and a few are based on recent performance (e.g., collision rates and transit on-time performance). The measures for this analysis evolved from a framework of common corridor objectives. The great strength of this analysis is consistency: virtually all types of investments can be evaluated based on a consistent set of performance measures.
• The Corridor Benefits Analysis will consider the interactive effects of major proposed investments at the corridor level. Measures selected for this analysis include user benefits (mainly the value of travel time savings), accessibility, and emissions and VMT. This type of analysis is best suited for large-scale capital expansion projects and some system management projects that can be evaluated through travel demand modeling.

The performance information, along with project costs, will be available as projects are selected for inclusion in the financially constrained long range plan. In addition, MTC will conduct a program-level performance assessment of investment alternatives through the long range plan environmental impact report.

Program Management
MTC uses performance measures to track the success of customer service projects operated by MTC in conjunction with partner agencies. The annual Project Performance Report describes project objectives, current performance and expectation of future performance for eight customer service projects: the Bay Area's roadside call box program, Freeway Service Patrol roving tow trucks, Regional Rideshare Program, Regional Transit Information System, TravInfo� traveler information system, TransLink� universal transit ticket, Pavement Management Technical Assistance Program, and Traffic Engineering Technical Assistance Program. Project performance criteria address customer satisfaction in addition to usage and project effectiveness.

Metropolitan Washington Council of Governments

Studying and accounting for current and future measures of transportation system performance has been a long-standing practice at MWCOG. Much of that has been associated with periodic surveys to sample changes in system performance, while more recent work has sought to develop: (a) a planning-oriented set of measures of effectiveness, and (b) an operations-oriented set of performance measures.

Transportation System Monitoring Surveys: Over the years a significant share of the annual budget of the Transportation Planning Board (TPB) of MWCOG has gone towards conducting periodic sample surveys of the use and performance of different aspects of the overall transportation system, with different aspects being sampled in different years. Each survey type has been conducted and refined a number of times, which has resulted in effective trend assessments of that aspect of system performance. Taken together they tend to give a fairly complete but not necessarily comprehensive picture of system performance over time. The following types of surveys have been conducted:

• Metro Core and Beltway Cordon Studies: from the early 1980's that count vehicles, transit riders, and estimate vehicle occupancy entering the downtown area or crossing the Beltway.
• Traffic Quality on the Metropolitan Washington Area Freeway System: Low-level aerial photography of traffic density and speed sampled in 1993, 1996, 1999, and 2002.
• Arterial Highway System Performance: GPS travel time and speed samples for about 45 arterial corridors, about 15 per year since 2000; with the cycle starting over again in 2003.
• High Occupancy Vehicle Lane Performance: initially just for Shirley Highway (I-395) in Northern Virginia, but expanded in later years to cover each of the other HOV lanes.

Regional Mobility and Accessibility Measures of Effectiveness: Recent work in updating the latest Long Range Transportation Plan developed and applied a set of measures of effectiveness to assess current and future system performance grouped in categories, such as: land use, miles of travel, energy consumption, mobile source emissions, mode shares, highway and transit congestion levels, accessibility, as well as information about safety, imperiousness, and freight.

Developing Performance Measures for Management and Operations: In early 2001 the TPB directed that performance measures related to management and operations be developed. A White Paper and Pilot Study was commissioned that presented examples of possible measures using currently available data as well as new data sources, while researching five categories of performance measures: (1) system quality, (2) reliability of travel, (3) safety, incidents, and enforcement, (4) impacts of transportation management centers and traffic signal systems, and (5) customer satisfaction. A regional subcommittee was working on selecting specific measures in Sept 2001. At that time, further work on this important topic was put on hold due to the urgency of addressing regional emergency preparedness planning and plans. The staff working on the performance measurement activities were also the ones tasked with the challenging new assignments related to Homeland Security. The development of informative operations-oriented performance measures and indicators continues to be an important goal. The FY04 UPWP calls for the development and refinement of performance measures, costs, benefits, and evaluation information for management and operations-oriented regional transportation planning.

Minnesota Department of Transportation

The Minnesota Department of Transportation (Mn/DOT) began developing system performance measures in the early 1990s. It now has measures, targets and regular reporting systems to help guide its Strategic Plan, 20-year Statewide Transportation Plan, and budget process.

Some of the earliest applications of measures were by maintenance engineers, such as for snow and ice removal, pavement markings, and for customer satisfaction. Planning measures for pavement, bridges and safety were first incorporated in the Statewide Transportation Plan in 1998. The new 2003 transportation plan is multimodal and fully performance-based. It has a comprehensive set of policies, outcomes, measures and performance targets as its framework. These measures will guide district and modal long-range plans as well as department-level action.

Statewide plan measures support ten policy directions:

1. Preservation of infrastructure	6. Regional Trade Center mobility
2. Right-of-way protection	7. Safety and security
3. System operations management	8. Internal management and program delivery
4. Modal transportation options	9. Public involvement
5. Interregional corridor mobility	10. Environment and community values

Mn/DOT's statewide plan measures are being used to assess the level of investment required to meet targeted performance levels over the next 20-years. Districts are estimating resource needs to close the gap between expected performance levels and targeted levels.

In addition, measures encompassing all major products and services are used in Mn/DOT's biennial business plan and budgeting process. Quality and process-oriented dashboards are used for project management, operations, and administrative services. Typically they are real-time measures with monthly or quarterly reporting.

Some of the fundamental concepts in Mn/DOT's practice are:

• Integration. Tie measures to existing planning and decision-making processes.
• Targets. Setting numeric performance targets is essential to driving action.
• Service Levels. Targets describe desired levels of service to customers.
• Gap Analysis. Targets define the gap to be addressed with new or redeployed resources, new strategies, or process improvements.
• Projections. Forecasting future performance levels can enhance planning by identifying the likely magnitude of future gaps.
• Hierarchy of Measures. To effectively manage using measures, operational levels of the organization often require process or output measures. These should support system outcome measures.

In the future Mn/DOT would like to advance its practice to a higher level in several areas:

• Modeling. Improve modeling capabilities to forecast future performance gaps.
• Tradeoff Analysis. Develop cost measures to facilitate scenario building and tradeoff decisions among capital investment options or among products and services. Groundwork is being laid to do this.

In conclusion, Mn/DOT believes it has built a stable, comprehensive framework of measures linked to department priorities. Nevertheless, the process of learning how to effectively manage with performance information and targets throughout the organization will continue.

Montana Department of Transportation

Like most state transportation agencies, performance measurement has multiple functions at multiple levels within the Montana Department of Transportation. Very briefly, performance measurements are used to communicate, track system condition, track program delivery, set goals and standards within business processes, report to decision-makers, and allocate resources. In general, the functional applications for performance measurement at the MDT are described below.

COMMUNICATION. Reports are published annually to track system performance (pavement condition, congestion, bridge functionality and condition) usage, and the status of actions committed to in the state's long-range transportation plan. During the last legislative session, staff used the predictive capabilities of the performance programming process to demonstrate the impact of transferring $37 million out of the state's long-range construction program. This was a powerful communication tool that helped to protect the state's highway construction resources.

PROGRAM TRACKING. The Montana Legislature requires performance measures for every major program area. These are reported on every quarter to the Governor's Budget Director. Examples of those for the MDT Planning Division include: 0% program lapse, substantial progress on 95% of the actions identified in the state's long-range multimodal transportation plan, and that 90% of the projects entering the STIP are consistent with the Performance Programming Process (P3 - described below).

The MDT also tracks construction program delivery on a quarterly basis. The specific metrics tracked include: percentage of the targeted construction program delivered, the percentage of all available authority obligated, and the percentage of targeted project types let to contract (reconstruction, rehabilitation, repave, bridge, miscellaneous). The metrics for program delivery were established through a negotiation with the contracting community based on their desire for greater program predictability, but the department maintained that the percentage of the program within the different construction work types would be generated from P3 based on the best distribution to address system needs within predicted budget constraints.

FUNDING ALLOCATION. The MDT allocates roughly 70% of its capital program through a process known as performance programming. This process uses management system information to iterate various improvement packages for the sixth year of the tentative construction program. Once established, the allocation is used to guide the projects entering the future construction program. This process attempts to keep the Interstate, non-Interstate NHS, and State Primary within performance targets by distributing an estimated future revenue stream to districts, systems, and types of work (reconstruction, rehabilitation, repaving, and bridge). To date this data intensive, and needs based distribution has survived two administrations and three legislative cycles. Staff is nearing completion of in-depth documentation to ensure process continuity in an era of staff-turnover.

Ohio Department of Transportation

ODOT has been developing and refining transportation performance measures for several years. Some of the early efforts included establishing inventories and condition assessments for pavement and bridges to activity cost accounting and qualitative measures for highway maintenance operations. These efforts were often done within the "silos" of the department and lacked consistency in their structure and use. Four years ago an effort was made as part of the quality initiative to identify key processes within the department and to develop meaningful measures and performance goals.

The Organizational Performance Index (OPI) was developed to monitor the overall performance of The Ohio Department of Transportation (ODOT) in key areas of focus. The measures that were developed have been used to measure ODOT's performance as well as the 12 District office's performances against the overall statewide average. The OPI system is continuously being refined to assure that the most accurate metrics are being applied in the measurement of the department's performance.

Currently the OPI includes measures for construction management, contract administration, equipment and facilities, finance, information technology, maintenance, plan delivery, quality and human resources, system conditions and traffic.

Each of these categories has a family of measures to measure several key parameters such as timeliness, cost, effectiveness, quality or compliance with department policies. The measures are evaluated and reported monthly through a formal review process.

The OPI reporting Web site has been established to quickly disseminate OPI reports and data to the divisions, districts, and general public in an organized fashion. All of the reports contained within this Web site are generated in the Adobe Portable Document Format (PDF). These automated reports are broken down into executive reports, topic reports, district reports and measure reports.

The Ohio Department of Transportation also publishes various transportation performance measures in the biennial Business Plan and the State of the Transportation System reports. These performance measures are also being carried forward into various planning documents such as the 30-year Access Ohio long-range transportation plan.

Southern California Association of Governments

In the preparation of the last 3 transportation plans, we have utilized performance measures to assist the elected board in making choices on investments both at the corridor and system level. SCAG's performance measures capture the relationship between transportation and a diversity of public policy concerns. They are specific, quantifiable, and easily understood, and they better inform elected officials and policy makers of the broad array of choices--and implications--for investing public and private transportation funds. This performance-based approach to planning provides a more inclusive process that allows input from a wide range of stakeholders, helps foster decisions that reflect better planning through a better understanding of the issues, and leads to more cost-effective investments.

The transportation plan adopted by SCAG in 1994 focused on congestion and air quality using performance measures such as vehicle miles traveled, average speed, mode split, average vehicle occupancy, and pollutant emissions. Following the 1994 plan, SCAG initiated a process to further develop its performance measures, involving a peer review committee of nationally recognized experts as well as public outreach to elected officials and stakeholders in the region.

By the adoption of the next plan in 1998, SCAG had expanded its performance measures to include mobility, accessibility, reliability, safety, air quality, cost-effectiveness, equity, and livable communities. The equity measure examined how different income and ethnic groups fared with respect to overall plan benefits and costs. The livable communities measure assessed how well the plan reduced automobile trips and automobile travel.

In the 2001 transportation plan, the equity/environmental justice analysis was expanded to address impacts of aviation system expansion. SCAG recognized the need for further articulation and evaluation of growth, the relationship between transportation and land use, and livable communities strategies. Accordingly, we initiated a regional growth visioning effort guided by our elected board and a Citizens' Advisory Committee to develop a comprehensive, far-sighted vision of how the region should grow. Out of this effort, a set of Growth Principles for Sustaining a Livable Region have been developed, along with criteria for evaluating growth scenarios. These criteria go beyond the traditional measures in addressing land use and its interface with the multimodal transportation system.

In addition to considering these new criteria in the development of the 2004 RTP, we have already added and improved a number of performance measures that look at the transportation system's productivity, preservation, and sustainability. These new measures address transportation solutions that may not always receive the most attention but which can provide substantial benefits for relatively little cost. While quantifying some of these measures may pose technical challenges, we recognize their importance in informing a truly long-range planning effort.

TriMet and the Portland, Oregon Region

TriMet Operational Measures:
Monthly Performance Report: This 12-page monthly report has been in place for over 20 years and presents a wide range of statistics pertaining to operating efficiency and ridership trends for bus and light rail. The report is presented to the TriMet Board of Directors each month and is available though the TriMet website. In addition to presenting trends month-to-month and year-to-year, it measures results against Objectives or Key Indicators for certain values.

Department-Specific Performance Reports: More detailed performance data is summarized in reports for specific departments with a focus on the operating departments. Time loss statistics are a primary example. The Capital Projects and Facilities Division produces a report that tracks individual projects against schedules and budget and other project management indicators.

Database Systems including the BDS Data Display (BUDS): Various databases are employed to perform customize analyses and evaluations. One names BUDS makes available the extensive bus dispatch system data (GPS-based data collection) for both tabular, graphical and map presentation. This data can identify ridership patterns, study service utilization, filter customer complaints and modify service or facilities accordingly.

Financial Analysis and Forecast Report: This report have been enhanced over the past 25 years to provide a comprehensive projection of TriMet expenditures and revenue projections in sufficient detail to facilitate the management of the Capital Improvement Program and service development plan. While not explicitly a performance report, it provides a macro-level assessment of the District's fiscal performance and capacity.

Regional Performance Measures:
Regional Transportation Plan: The 2000 RTP expands the use of congestion as a performance measure to be included as regional policy. The "Areas of Special Concern" designation allows for a broader definition of performance in mixed-use centers and corridors, where transportation solutions solely aimed at relieving congestion are inappropriate for functional, physical, financial or environmental reasons. This expansion of performance measures for all modes will continue with future RTP updates. While congestion should be factored into a more diverse set of measures, it should be evaluated in a more comprehensive fashion to ensure that transportation solutions identified in future RTP updates represent the best possible approaches to serving the region's travel demand.

Regional Growth Management Performance Measures: The region's MPO, Metro, first published the Performance Measures Report in March 2003, evaluating the effectiveness of Metro's regional growth management policies. The methodology established eight Fundamental values and over 100 performance indicators – and analysis of 80 of them. Those measures are now being reviewed and prioritized and potentially reduced to improve their presentation. The fundamentals address the strength of the local economy, efficient land use within the Urban Growth Boundary, environmental protection and restorations, development of a balanced transportation system, preserving the integrity of neighboring cities, improving the sense of place within the growth boundary, ensuring diverse housing options and creating a vibrant place to live and work.

Washington State Department of Transportation

Accountability is a top priority at WSDOT. The agency produces a quarterly performance report: Measures, Markers and Mileposts; also called the Gray Notebook that provides a comprehensive account of agency and system performance using a unique performance journalism approach. The Gray Notebook is the basis for all performance assessment and reporting as well as public and legislative communication. A comprehensive index of all published performance information can be found at http://www.wsdot.wa.gov/accountability/graybookindex.htm; the most recent edition of the Gray Notebook can be accessed at http://www.wsdot.wa.gov/accountability/graynotebook.pdf.

What is New at WSDOT?
The most current efforts in implementing WSDOT's "what gets measured, gets managed" mantra include the development and application of enhanced congestion and reliability performance measures using ITS data; the implementation of statewide transportation benchmarks; and the new comprehensive project reporting process for the recently passed 2003 Nickel Transportation Funding Package.

Development of Statewide Benchmarks
The Washington State Transportation Commission and WSDOT, working collaboratively with the Washington State Transit Association, implemented statewide benchmarks for eight categories of transportation system performance as well as one organizational measure. In doing so, the Commission responded to the policy and benchmark goals set forth in legislation that was passed during the 2002 session (RCW 47.01.012). The Transportation Benchmarks web page provides access to all relevant documents and performance measures that were implemented and recently adopted by the Commission. As benchmarks are refined and updated, they will be posted to this web site. The site contains a link to the complete Transportation Benchmarks Implementation Report, as well as an executive summary: http://www.wsdot.wa.gov/accountability/benchmarks/.

Measuring Congestion and Reliability using ITS Data
In May of 2002 WSDOT launched an aggressive effort to enhance the use and development of new congestion performance measures that are based on real time travel time information versus modeled data. A major emphasis was the deployment of a significantly enhanced Incident response program and the assessment of operational and capital improvements on non-recurring and recurring congestion. Please access the following website for a complete account and links to active websites: http://www.wsdot.wa.gov/accountability/peaktime/WSDOT_Measuring_congestion.pdf.

Enhanced Project Delivery Reporting: The Beige Pages
A new feature in the June 2003 edition of the Gray Notebook, published in August, is the Beige Pages which represent a major effort in project reporting that reflect the "no surprises" rule for delivery of the recently passed 2003 Transportation Nickel Funding Package. It is a web linked, quarterly report for the legislature and public that contains extensive, detailed information on projects under way as well as heads up info on upcoming projects and issues. To directly access the Beige Pages, beginning on the fist page of the Gray Notebook, paste the following link into your web browser (this direct link doesn't work in Microsoft Word format): http://www.wsdot.wa.gov/accountability/GrayNotebook.pdf#page=5, or go to the current Gray Notebook http://www.wsdot.wa.gov/accountability/graynotebook.pdf and skip to the Project Reporting section on pdf page 5.

U.S. Department of Transportation

DOT's Interest in Encouraging
Transportation System Performance Measurement
In Statewide and Metropolitan Planning

• A number of Federal planning and budgeting systems over the past 40 years have encouraged the use of performance measures to evaluate the effects of Federal programs. Examples include PPB, zero based-budgeting, and management by objectives.
• The Government Performance and Results Act of 1993 (GPRA) established a government–wide system of strategic planning and performance measurement that applies to almost all Federal agencies and their programs.
• The objective of Congress in passing this law was to promote performance-based decision-making by agencies, with the following intended results:
• Heightened attention to the outcomes of regulatory and investment decisions;
• Strengthened measurement and analytic capabilities to quantify those outcomes; and
• Improved program effectiveness and public accountability by promoting a focus on results, service quality, and customer satisfaction.
• DOT is encouraging the use of transportation system performance measures by State/local decision-makers, acting through the metropolitan and statewide planning processes, to evaluate their highway and transit investment programs.
• There has been similar interest in performance measures for similar reasons in other countries and in the United States at all levels of government.
• DOT's surface transportation reauthorization proposal, SAFETEA, includes a number of proposals that involve the use of performance measures.
• For the transit program, we propose a ridership-based Performance Incentive Program, using up to ten percent of FTA's Urbanized Area and Rural program funds.
• For the highway safety program, we propose three performance grant programs where funding would be awarded based on highway safety performance measures such as alcohol-related fatalities and seat belt use.
• We propose a $135 million a year ITS Performance Incentive Program that would award funds based on State performance on various ITS criteria.
• We propose a Surface Transportation System Performance Pilot Program in which up to five states would be given the flexibility to manage their highway programs on a systematic, performance basis, without regard to the programmatic lines in which the Federal-aid highway program is normally structured.
• With regard to the use of transportation system performance measures in planning, SAFETEA:
• "Encourages the continued improvement and evolution of the metropolitan and statewide transportation planning processes by metropolitan planning organizations, State Departments of Transportation, and public transit operators through the use of performance-based approaches in the development of transportation plans and investments as guided by the planning factors identified in …(Title 23)"
• Establishes minimum expenditures for improving the quality of collection and reporting of strategic surface transportation data to provide critical information about the extent, condition, use, performance and financing of the system.

• We hope to draw upon the experience of experts and practitioners in the area and focus on lessons learned in the application of transportation system performance measures in State and local planning. We want to gauge the state of existing practice, discuss how it might be improved, and get ideas from practitioners on an agenda for the future.
• We are working with the Transportation Research Board and major stakeholder organizations to sponsor discussions of the value of performance measures, how they are being used to assess the overall transportation system as well as the planning and decision making process and how to encourage increased use of performance measures.
• The Department of Transportation believes that a collaborative approach that builds on the considerable work already done by others in a number of venues represents the best strategy to encourage the use of transportation system performance measures in state and local planning. This is why the Office of the Secretary of Transportation, FHWA, and FTA are working together through the Transportation Planning Capacity Building Program to hold this meeting.

Appendix C
Index of Resource Papers

Binghamton Metropolitan Transportation Study

• Long Range Plan, Transportation Tomorrow: 2025: http://www.gobroomecounty.com/departments/TransportationTomorrow2025.pdf

Maryland DOT

• 2003 Annual Report: http://www.e-mdot.com/State%20Report%20on%20Transportation/Documents/Annual%20Attainment%20Reports/2003MarylandAttainmentReport.pdf
• 2004 Annual Report: http://www.e-mdot.com/State%20Report%20on%20Transportation/Documents/Annual%20Attainment%20Reports/Final%202004%20Attainment%20Report.pdf

Montana DOT

• Strategic Business Plan: http://itsd.mt.gov/stratplan/MontanaStrategicPlan0402.pdf.
• Performance Programming Process (P3): http://www.mdt.mt.gov/.

Ohio DOT

• Statewide long-rang transportation plan, Access Ohio: http://www.dot.state.oh.us/planning/File%20Directory/AccessOhio.htm
• Organizational Performance Index (OPI): http://www.dot.state.oh.us/dist12/PDF%20files/info%20and%20analysis.pdf
• Ohio's State of the Transportation System and 10-Year Plan (power point presentation): http://www.dot.state.oh.us/divplan/pdfs_files/sos2003b.ppt

Southern California Association of Governments

• State of the Region 2002: http://www.scag.ca.gov/publications/index.htm

TriMet

• MAX System Overview: http://www.trimet.org/inside/history/maxpage.htm#ridership

Washington State DOT

• The Gray Notebook- Quarterly Report on Performance Measures: Measures, Markers and Mileposts: http://www.wsdot.wa.gov/accountability/graynotebook.pdf