Certification Checklist for Travel Forecasting Methods

Introduction

Federal transportation planning legislation requires each metropolitan planning organization (MPO) to develop a transportation plan as part of its planning process [23 U.S.C. 134 (g) and 49 U.S.C. 5303 (f)]. This transportation plan must cover at least a 20-year planning horizon, and "shall include both long-range and short-range strategies/actions that lead to the development of an integrated intermodal transportation system that facilitates the efficient movement of people and goods" [23 CFR 450.322].

A transportation plan requires valid forecasts of future demand for transportation services. These forecasts are frequently made using travel demand models, which allocate estimates of regional population, employment and land use to person-trips and vehicle-trips by travel mode, route, and time period. The outputs of travel demand models are used to estimate regional vehicle activity for use in motor vehicle emissions models for transportation conformity determinations in non-attainment and maintenance areas, and to evaluate the impacts of alternative transportation investments being considered in the transportation plan.

The Transportation Conformity Rule established a regulatory requirement that includes minimum specifications for travel models used to forecast vehicle activity for regional emission analyses in conformity determinations in certain non-attainment and maintenance areas [40 CFR 93.122 (b) and (c)]. However, these minimum specifications apply only to metropolitan planning areas with an urbanized area population over 200,000 that are also serious, severe or extreme ozone or serious carbon monoxide non-attainment areas. All other non-attainment or maintenance areas must continue to meet the minimum specifications for travel models established in the Conformity Rule to the extent that those procedures have been the previous practice of the MPO.

Although there are no other requirements that travel demand models must be used in the metropolitan transportation planning process, the travel forecasting methods used by an MPO should be addressed in the certification review to ensure that they adequately support the applications for which they are being used. These applications can vary considerably from one MPO to another, depending on such factors as non-attainment status, regional population and economic growth, and the types of strategies/investments being considered in the transportation plan.

The questions included in this checklist are designed to provide the certification review team with an overview of the travel forecasting methods being used by an MPO, the suitability of those methods for intended applications, and the technical capabilities of the planning staff in applying the methods. In those cases where responses to the checklist questions raise serious concerns on the adequacy of the forecasting methods, the certification review team should request a more in-depth review by FHWA Resource Center or FHWA/FTA Headquarters travel model experts.

Key Indicators of Risk

Determining the adequacy of an MPO's travel forecasting methods begins with an understanding of how the forecasts will be used. Where forecasts are used to estimate motor vehicle emissions for transportation conformity determinations, or where they are used to evaluate major transportation investment alternatives such as new highways or transit lines, the forecasting methods are more likely to be scrutinized. Federal findings (e.g., Conformity Determinations or Records of Decision) based on weak or poorly documented travel forecasting methods may be susceptible to legal challenges. Such challenges can, at a minimum, lead to time consuming legal proceedings and delays in project implementation. In some cases, courts have even overturned federal findings because of inadequate forecasts, resulting in costly supplemental analyses and additional project delays. Adverse court decisions also establish legal precedent, which can be used by plaintiffs in subsequent challenges against other transportation agencies.

The following questions probe whether an MPO's forecasting methods are more likely to receive close scrutiny from other agencies or outside advocacy groups.

• Is the metropolitan area a designated serious, severe or extreme ozone or serious carbon monoxide non-attainment area?Metropolitan planning areas with an urbanized area population over 200,000 that are designated as serious, severe or extreme ozone or serious carbon monoxide non-attainment areas must meet certain minimum travel modeling requirements as specified in the Transportation Conformity Rule [40 CFR 93.122 (b)]. Failure to meet these requirements may result in a delay in conformity determination, or even a conformity lapse and the restriction of federal highway and transit funds to the metropolitan area.

• Is the metropolitan area a designated non-attainment or maintenance area, and has the MPO used travel demand models previously? Pursuant to 40 CFR 93-122 9(c)), if an MPO in a non-attainment or maintenance area currently uses or has used a travel demand model, then they must continue to use a model with similar or greater sophistication for regional emissions analysis in transportation conformity determinations. If the non-attainment or maintenance area does not have a history of travel demand model use, there are other prescribed minimum requirements for reasonable estimation of VMT growth.

• Does the metropolitan area plan to apply for an FTA transit new start grant? The FTA New Starts Program is a discretionary grant program with its own requirements for assessing the costs and benefits of proposed alternatives. MPOs that apply for a new start grant must use travel forecasting methods that meet FTA requirements.

• Does the transportation plan include any major projects that will significantly increase highway capacity? Regionally significant highway projects (e.g., new highways or additional lanes on existing highways) have been consistently targeted by national environmental advocacy groups as contributing to "urban sprawl" and "induced demand." These projects are particularly susceptible to legal challenges in which the plaintiffs hire their own travel model experts to dissect the forecasting methods used to derive forecasts of future traffic.

• Is the metropolitan area proposing any transportation projects where there is strong and coordinated opposition by local advocacy groups? Local groups with sufficient resources, or in coordination with national organizations, may also hire their own travel model experts to challenge controversial projects on methodological grounds.

• Has the MPO been a defendant in, or threatened with, legal action in which the adequacy of their travel forecasting methods was challenged? If so, what was outcome of this action? MPOs whose travel forecasting methods have been challenged in the past may be vulnerable to future challenges. However, if the challenge was summarily dismissed, or if the travel forecasting methods were upgraded in response to identified deficiencies, the MPO may actually be immunized against future challenges.

Affirmative answers to any of the above questions indicate that the travel forecasting methods used by the MPO are likely to be scrutinized by travel modeling specialists working on behalf of agencies or organizations other than the MPO.

Key Indicators of Agency Technical Capabilities

The certification review team can obtain a general overall assessment of the technical capabilities of the staff responsible for developing and applying the travel forecasting tools used by an MPO by looking at a few key indicators. These indicators are covered in the following questions.

• Who is responsible for travel forecasting at the MPO? Technical staff with expertise and experience in travel demand models is needed to develop, maintain and interpret the output from travel forecasting methods used in metropolitan transportation planning applications. This expertise may be provided by MPO in-house staff, by technical staff from another agency (e.g., another MPO or the state DOT), or by outside contractors.

If another governmental agency provides required modeling technical expertise:

• Is there a formal memorandum of agreement between the agencies to delineate technical responsibilities, lines of communication and review, authorized expenditures and reimbursement procedures?Without a formal agreement, the MPO must rely entirely on the generosity of other agencies to provide the appropriate expertise and tools.

If contractors perform all travel model development:

• Who, if anyone, on the MPO staff is responsible for evaluating the technical work of the contractor? Even if contractors develop the travel demand forecasts, some in-house expertise is still needed in order to independently evaluate the reasonableness of the travel forecasts produced, to defend the methodology in public forums, and to provide institutional memory of what changes were made to the methodology or why they were made.

If in-house staff actively participate in model development and application:

• What formal training has the MPO technical staff received in travel demand forecasting? Formal training may include coursework taken as part of an academic degree program, or completion of one or more professional training courses offered by FHWA or FTA.
• Does the MPO technical staff require training in specific technical areas? If no one on the MPO technical staff (where technical staff may be provided by another MPO or State DOT) has formal training or experience in the use of travel forecasting methods, the MPO cannot assess the adequacy or validity of its long range travel demand forecasts. These MPOs may be unable to adequately defend their forecasts against technical challenges to the models or planning assumptions.

• Does the MPO organizational structure include a technical committee to review planning assumptions and forecasting methods?Many MPOs establish technical committees comprised of state and local transportation planning professionals, private consultants, and other individuals having interest in the forecasting process. Such technical committees can help mitigate potential challenges by providing an early opportunity for public participation in the technical process, by de-mystifying the forecasting methodology, and by encouraging broad consensus in developing key planning assumptions.

• Does the MPO have a strategic plan and a guaranteed minimum level of funding in its Unified Planning Work Program (UPWP) for maintenance and improvements to its travel forecasting methods?MPOs that have a well-defined and adequately funded program for data collection, and for travel model maintenance and enhancement, are more likely to have a technically sound forecasting process. By contrast, MPOs with no specific improvement plan or regular source of funding are more likely to base their forecasts on outdated data and methods that are not consistent with current state-of-the-practice.

• Has the MPO convened a peer review or other independent assessment of their travel forecasting methods? A peer review can effectively diagnose deficiencies in an MPO's travel forecasting methods, and can inoculate the MPO against frivolous legal challenges, if improvements recommended by the peer review are actually implemented. Alternatively, unimplemented recommendations provide an obvious target for legal challenges. In general, peer reviews provide good indicators of the MPOs' commitment to its travel forecasting technical process.

If a peer review was convened, the following information should also be obtained:

• The date of the most recent peer review
• The stated purpose of the peer review
• A list of participants
• Recommendations arising from the peer review
• The MPO's plan and/or schedule to address the peer review recommendations

Documentation

Most of the questions raised with respect to travel forecasts can be addressed by adequate technical documentation of the input assumptions and the methods used to develop the forecasts. In addition, federal transportation planning regulations require such documentation [23 CFR 450.316 (d)].

The Certification Review Team should request and obtain readily available written, technical documentation from the MPO covering the following subject areas:

1. An inventory of the current state of transportation in the metropolitan area.
2. Key planning assumptions used in developing the forecasts.
3. Descriptions of the methods used to develop forecasts of future travel demand.

Each of these subject areas is described more fully below.

1. Inventory of Current Conditions

The foundation for any forecast is a comprehensive and objective inventory of current conditions with respect to both transportation supply and demand.

The inventory documentation should include the following summary measures for the metropolitan planning area:

• Highway system - total centerline and lane-miles of roadway by functional class
• Transit system - total route miles of transit service by mode (e.g., bus vs. light rail)
• Other transport modes (as appropriate) - pedestrian and bike paths, ferry service, etc.
• Population - total population and households, and their geographic distribution within the study area
• Employment - total number of jobs, and their geographic distribution within the study area
• Vehicle miles of travel - average daily and annual VMT by highway functional class
• Transit use - systemwide transit ridership and share of regional trips made on transit (average daily and peak)
• Congestion - description and duration of peak period (i.e., what criteria distinguish peak vs. off-peak travel (e.g., highway level of service?))
• Land use - amount and geographic distribution of total land area that is currently developed, available for development, or not developable.
• Special conditions - any unusual characteristics of the study area that significantly impact overall travel volumes or patterns (e.g., high tourist area, major intermodal port, heavy truck through traffic, state capital)

The data sources for summary measures should be identified, including descriptions of their currency and frequency of updates. Data sources that are significantly out-of-date should be identified as candidates for updating in future UPWPs.

Data on highway VMT and congestion summary measures should be consistent with, and/or derived from traffic monitoring data used in the TMA's Congestion Management System (CMS).

2. Planning Assumptions

The principal determinants of any long-range travel demand forecast are the planning assumptions about the growth and distribution of population, developed land, and individual travel preferences. If these assumptions are not consistent with statewide or regional controls or with past trends for the study area, they need to be explained and justified.

In non-attainment and maintenance areas, planning assumptions should be consistent with the joint FHWA/FTA/EPA Guidance on the Use of Latest Planning Assumptions in Conformity Determinations (January 18, 2001). This guidance is available at http://www.fhwa.dot.gov/environment/cnfplngg.htm.

The documentation of planning assumptions should, at a minimum, address the following expected changes in the study area:

• Population change - expected change in regional population over the duration of the Transportation Plan. Population assumptions should be compared to past trends, and to statewide demographic control totals, where available.
• Employment change - expected change in regional employment over the duration of the Transportation Plan. Employment assumptions should be compared to past trends, and to statewide economic growth control totals, where available.
• Regional distribution of future population, employment and land use - the procedures used to allocate future population, employment and other activity generators within the metropolitan area. Are the land use forecasts consistent with local jurisdictions' Master Plans? If land use models were employed, these should also be documented under forecasting methods.
• Demographic changes - changes in the demographic characteristics of the study area population that would significantly impact aggregate tripmaking behavior and/or travel patterns. Demographic changes might include, auto ownership, household income, household size, multi-worker households, minority households, etc.
• Travel behavior changes - changes in the tripmaking behavior of travelers and households that would significantly impact aggregate tripmaking behavior and/or travel patterns. Travel behavior changes might include telecommuting, Internet shopping, trip chaining, etc.

Updates to the transportation plan should compare current population, employment and demographic characteristics with forecasts made in previous plan updates. Significant differences between previous forecasts and current conditions should be documented and explained, and assumptions should be revised, accordingly.

3. Forecasting Methods

The complexity of a study area's forecasting methods can vary considerably, depending on current transportation conditions, and on the future transportation investments and policies being evaluated. For example, an MPO with limited public transportation service and few or no choice riders may be able to use a simplified, off-model approach to estimate transit mode share, unless it plans to evaluate major public transit investments as part of its transportation plan. Alternatively, an MPO that plans to make significant investments in operational technology (e.g., areawide signal synchronization, ramp metering, etc.) may need to add a traffic micro simulation model to its model set.

The technical documentation of the travel forecasting methods or models should include the following information:

• Last model revision - when (what year) was the current set of travel models last revised (e.g., new variables, new model algorithms, recalibrated using new data)?
• Model specification - description of models used (e.g., gravity vs. destination choice) and interactions between models, specification of key model coefficients, calibration results (e.g., goodness-of-fit measures).
• Calibration data - what data was used to calibrate the model set (e.g., local home interview survey, national surveys (e.g., NHTS, CTPP), models "borrowed" from another urban area)? How current is the data source?
• Local survey - if a local home interview survey was used to calibrate the model, when (what year) was the survey conducted, how many valid household records were collected?
• Model validation - what year and data source was the model validated against?
• Size of network - how many links are in the model highway network; what highway functional classes are included as network links; has a compatible transit network been developed?
• Number of zones - How many transportation analysis zones (TAZs) are included in the model?
• Non-home based travel - How is non-home based travel modeled (e.g., freight, commercial services, through traffic, tourists)?

The technical documentation should be readily available to all interested parties, consistent with the public involvement provisions in the planning regulations [23 CFR 450.316 (b)(1)]. Technical documentation should be updated on a periodic basis to reflect changes in the models or the key planning assumptions used to develop the transportation plan.

MPOs that cannot provide written technical documentation, or whose documentation does not adequately cover the above subject areas, are vulnerable to legal challenges even if their planning assumptions and forecasting methods are otherwise satisfactory.

Suggested Actions by the Certification Review Team

MPOs that are able to provide adequate documentation of their forecasting methods and assumptions, and that have generally positive indicators of technical capabilities and low indicators of risk, require no further action by the certification review team.

The absence of any technical documentation, or documentation that does not adequately address key subject areas should be discussed as a recommended area for improvement during the certification review.

Indications of weak technical capabilities with respect to travel forecasting methods should also be discussed as a recommended area for improvement during the certification review. MPO's that have not had their travel forecasting methods recently peer reviewed should be encouraged to convene a peer review. FHWA provides financial support to MPOs for technical peer reviews through its Travel Model Improvement Program (TMIP).

MPOs engaged in high-risk applications (e.g., conformity determinations or controversial highway projects), and with indications of weak technical capabilities should have their forecasting methods reviewed by FHWA/FTA travel model experts.