September/October 2004
The Uncertainty of Forecasts
by John S. Miller
When it comes to forecasting transportation demand over long time horizons, this author contends that some trends are more reliable than others.
Transportation planners often are asked to predict socioeconomic, demographic, and land use trends that will affect future
demand for transportation services. But legitimate questions immediately arise: How well can such trends be envisioned, in what areas are forecasts likely to be imperfect, and how can such uncertainties be imparted to decisionmakers? The impetus for
investigating the accuracy of predictions stemmed from background work conducted to support the development of VTrans2025,
Virginia's statewide, multimodal long-range transportation plan, begun in 2000 and scheduled for completion in 2005.
|
A lone vehicle travels down a Virginia highway at dusk. Accurately predicting future demand for transportation infrastructure like this highway can be challenging, as technological innovations and social or political developments can cause unforeseen shifts in trends. Photo: Ed Deasy, Virginia Transportation Research
Council. |
A century of national transportation
data suggests that predictions
over a long time horizon are not
equally accurate for all types of information.
A forecaster looking 25
years ahead at any point between
1900 and 1975 probably could have
predicted about half of the transportation-
related trends accurately at the
national scale. Predictions for socioeconomic
factors, such as population,
ethnicity, employment, income,
and household sizes, are generally
feasible, albeit imperfect, provided
the geographical area is adequately
large. Predictions for trends based
on technological innovation, social
change, or legislative factors, however,
are much more difficult.
Many themes in transportation
planning, such as modal split for
passenger and freight travel, land use
legislation, potential improvements
in technologies that would help
transportation operations, and public
willingness to support additional
transportation infrastructure, fall into
the latter category. Within long-range
transportation plans, one should
clearly indicate those factors that are
likely to be predicted accurately.
There might be other predictions
that are more difficult to get right.
Forecasting Trends Over a Long Horizon
Long-range transportation plans,
with horizons of 10 years or greater,
often are viewed as a process for
enabling decisionmakers to evaluate
the strengths and weaknesses of
various transportation alternatives.
These plans necessarily rely on a
variety of projections regarding the
future: How many people will live in
a region; how much will they earn;
what kinds of jobs will they have;
and where, when, and by what
mode will they travel?
To support the creation of VTrans2025, staff from the various modal agencies in Virginia-the Virginia Department of Aviation, Virginia Department of Rail and Public Transportation, Virginia Department of Transportation (VDOT), and the
Virginia Port Authority-asked the Virginia Transportation Research Council (a joint venture between VDOT and the University of Virginia) to identify key socioeconomic trends likely to affect transportation demand in 2025. The trends report, Expected Changes in Transportation Demand in Virginia by 2025 (available at http://virginiadot.org/vtrc/main/online_reports/pdf/03-tar5.pdf), is not the VTrans2025 statewide multimodal plan; rather, the trends report was designed simply to produce supporting information for developing the plan. This supporting information was expected to identify trends in four areas:
- Historical and projected socioeconomic
trends, such as population,
employment, and personal
income. For example, how will
the age of Virginia's population
change by 2025?
- Relevant changes in public
policy, legislation, and technology.
Specific topics include local
legislation on growth management,
improvements in traffic
operations, and Federal funding.
- Freight projections and changes in
market share for the various modal
freight movements. For example,
what will be the value of freight
shipped by the various modes of
air, rail, and truck by 2025?
- Passenger travel trends, including
mode choice, automobile ownership,
and changes in vehicle miles
traveled (VMT).
One key question that arose during preparation of the forecasts is, Why is it not possible to forecast all
trends for all Virginia jurisdictions,
down to the county and city levels,
equally well to year 2025? The State
desired a high level of detail for consistency
among the different agencies,
so that all trends would be forecasted
to the same level of geographic detail
and for the same horizon year. Another
reason for the high level of
detail was more fundamental: the
VTrans2025 Technical Committee
wanted to map transportation services
to expected transportation demand.
A legitimate question was
raised: If one can predict a statewide
population for the year 2025, is it also
possible to predict other trends, such
as the modal splits for passenger
miles traveled or tons of freight
shipped? Further, why not forecast
those trends not just at a statewide
level of detail but also at the city or
county level? To answer these questions,
researchers at the Virginia
Transportation Research Council
(VTRC) examined national-level data
to determine how well forecasting
attempts in the past would have predicted
current conditions.
A related question is the importance
of the horizon year 2025, established
by decisionmakers as the target
year. Even if predictions are feasible, is
it desirable to make projections to the
year 2025? One view is that because
transportation represents such a broad
set of phenomena, different elements
will have different planning cycles.
A second view is that it is more
important for planning horizons to
be consistent. For example, during a hearing on transportation and air
quality before the U.S. Senate's Committee
on Environment and Public
Works in July 2002, Federal Highway
Administrator Mary E. Peters noted
that air quality plans often cover 5
to 10 years, compared with the 20-
year horizon for transportation
plans. She also noted that some
stakeholders have suggested bringing
the planning horizons and frequency
of updates closer together,
either by lengthening the former or
shortening the latter.
A third response is that even longer horizons are necessary, because 20 to 30 years is a relatively short time frame for the infrastructure impacts of transportation on land use to take effect. Given these three responses, a longer forecast horizon
may facilitate more complete analysis of transportation and land use, provided that consistency among various types of plans can be achieved.
Types of Data Forecast to 2025
To support VTrans2025, trends and forecasts were developed across four main areas: socioeconomic trends, public policy changes, multistate freight requirements, and measures of transportation use. Socioeconomic trends-population growth, income and employment changes, and household size and location-are a reasonable starting point for any long-term plan since these factors affect how the State will evolve and are somewhat stable over time at the statewide level.
Sources of Demand Affect Measures of Transportation Use
|
The figure shows how various trends-socioeconomic, policy, and freight-feed into
predictions about transportation use. The dashed arrows signify potential feedback
between measures of transportation use, such as VMT, and trends such as home prices. |
Public policy changes in the areas
of national legislation, consumer
needs, and transportation technology
may significantly alter how
transportation services are delivered.
Multistate freight requirements
also influence transportation demand
because freight movements
can use Virginia's transportation
network or may bypass the State
altogether. These three categories-
socioeconomic changes, policy
changes, and freight changes-affect
sources of transportation demand.
And the way the transportation
system responds to these sources of
demand may be expressed as the
fourth category-measures of transportation
use-reflected by passenger
VMT, mode choice for passengers
and freight, tons of freight
shipped, and travel time.
Although these four areas are
presented as discrete sections for
ease of illustration, they are related.
Rising incomes, for example, generally
are associated with increased
travel. Rising home prices in a close-in
suburban county may cause some
residents to locate farther away from
their jobs, thereby increasing passenger
VMT. The resultant traffic congestion
may in turn cause prospective
home buyers to place a premium on
close-in suburban homes.
National Changes in Population and Automobile Ownership, 1907-2000
|
The graph shows national trends in population and
automobile registrations in the United States from 1907
through 2000. Both have risen since 1907 but at different
rates. Since approximately 1945, automobile registrations
have outpaced population growth. Researchers could
predict this steady increase fairly easily. |
Rules of Thumb for Forecasting
Generally, more faith may be held in
trends that are less susceptible to
sudden change, relatively large in
geographical scope or
based on a relatively
large data set, and projected
over a shorter
rather than a longer
horizon. For example,
2010 population forecasts
for the State
of Virginia are more
reliable than 2025 home
price forecasts for
Charlottesville (a small-to
mid-sized city southwest
of Washington, DC)
for three reasons:
- Historically, population
trends have
grown at a relatively
steady rate without
sudden increases or
decreases, whereas
home prices can
drop suddenly
because of market conditions,
changes in school quality, or
changes in an area's employment
outlook.
- Virginia is much larger than
Charlottesville; the likelihood of a
spurious trend emerging is much
greater for a small city than it is
for an entire State.
- The likelihood of an unforeseen
change occurring is greater over
the next 25 years than
over the next 10 years.
A fourth factor that
influences the ability to
make predictions is that
trends driven by market
or socioeconomic mechanisms
appear to be easier
to predict than those
driven by legislative fiat.
The continued decline in
agriculture-related employment,
for example,
can be forecast relatively
easily, since the increased
efficiency of farming
techniques and the
higher economic benefits
of land used for purposes
other than agriculture are
trends that are expected
to continue based on
market principles. In
contrast, projections of
land use trends based on
local zoning ordinances
or local plans are less
reliable, since they are
subject to change and
receive pressure from market forces, popular will, or political
interests.
A fifth factor is the quality of data
and the availability of multiple data
sources. Population data for the State
of Virginia, including forecasts, are
available from the U.S. Census Bureau
as well as private data sources. Other
types of data, however, are limited,
making forecasts more difficult.
Freight transport data, for example,
historically have been difficult to
acquire owing to the proprietary
nature of commodity flows and shipper
characteristics.
National Changes in Transit
Ridership, 1907-2000
|
This graph shows national trends in light rail, rapid rail,
and bus ridership in the United States from 1907 through
2000. Unlike the steady increase in automobile registrations,
the trends for streetcar/light rail, rapid rail transit,
and bus ridership have experienced periods of growth
and decline. Consequently, predicting trends for these
modes of transportation is more difficult. A trend line
forecast made in the early 1900s for the use of these
technologies over this multi-decade horizon would have
had limited value. Source for both graphs: VTRC, using raw data
from Bureau of Transportation Statistics, U.S. Census Bureau,
American Public Transportation Association, and Saltzman, A.
"Public Transportation in the 20th Century." |
Case Study in Mode Choice
Forecasts for socioeconomic measures
such as population, income,
and employment in 2025 are readily
available at the metropolitan, State,
and national levels. Within the realm
of policy, however, forecasting precise
legislative, technological, and
social trends a quarter century into
the future generally is not possible.
A practical reason is that over a 25-
year horizon, identifying key social
responses that may result from technological
or organizational innovations,
economic changes, or political
events is impossible. Anecdotal examples
of unforeseen disruptions
include the increase in business applications of the Internet in the
1990s, the personal computer revolution
in the 1980s, the rapid rise in
purchases of television sets between
1947 and 1952, and the number of
persons educated under the G.I. Bill
following World War II.
The following case study in predicting
the modal split for passenger
travel suggests the difficulty of foreseeing
fundamental policy shifts. The
example suggests that envisioning
technological and social change is a
much more difficult task than extending
population or employment
trend lines.
Change in Transit Ridership, Population, and
Automobile Ownership Relative to 1925
|
This figure shows the change in transit ridership,
population, and automobile ownership relative to
1925. The five trends (population, automobile
registrations, streetcar/light rail, rapid rail transit,
and bus ridership) now are presented as ratios to
their 1925 levels. For example, in 1950, bus ridership
was about six times its level in 1925. On the other
hand, by 1950, streetcar ridership had dropped to a
fraction of its 1925 level. Source for both graphs: VTRC,
using raw data from Bureau of Transportation Statistics, U.S.
Census Bureau, American Public Transportation Association,
and Saltzman, A. "Public Transportation in the 20th Century." |
A century of data provides some
perspective on forecasting social and
technological developments related
to transportation. Looking backward
with the perspective of hindsight, A.
Saltzman writes in an article, "Public
Transportation in the 20th Century,"
in Public Transportation that the
trends that occurred are not surprising.
At the turn of the century and
peaking around 1920, for example,
streetcar ridership was strong, owing
to technological change (electrifying
horse railways) and land use
change (dispersion of cities).
The fact that public transportation ridership, including street cars, light rail, rapid rail transit, and bus, was lower in 1935 than in 1930, especially in light of increasing population (and no corresponding increase in automobile
registrations),
can be explained by an
economic change (the
Great Depression). Social
change (World War II)
explains the increase in
all public transportation
modes in the early 1940s,
whereas economic and
land use changes (increasing
incomes and greater
dispersion of cities)
may be reasons for the
automobile's subsequent
dominance.
In addition, Saltzman notes that the shift from a 6-day to a 5-day workweek may have contributed to
the near-demise of rapid rail transit, since that ridership historically benefited most from the commuter
trip. Since the 1970s, automobile ownership has continued to rise, but transit has stopped declining in raw numbers because of several possible reasons. Among them are continued population increases, State and Federal programs designed to increase
use of public transportation, higher parking and congestion costs in some metropolitan areas, and greater environmental
concerns.
Interestingly, it appears that the
large changes in the trends—between
1907 and 1950—were driven
strongly by technological, social,
economic, and demographic changes
as opposed to public policy initiatives
alone. Technological developments,
such as innovation in rubber-tired
vehicles that enabled the bus
to take market share away from the
streetcar in the 1920s, had more of
an impact on ridership trends than
later public policy initiatives, such as
encouraging the use of transit instead
of automobiles in the 1990s.
There are, of course, instances
where public policy initiatives have
had a marked influence, such as
the combination of vehicle, roadway,
and driver improvements that
have decreased fatality rates for
automobile passengers during the
past few decades.
Looking forward is much more
difficult than looking backward. For
example, if someone in the year
1925 had been looking ahead based
on previous data, what might he or
she have predicted over the next
two decades? What trends would a
national-level forecaster have identified
correctly? What trends might
have remained hidden?
Comparison of Predicted and Actual Changes Assuming a 1925 Base Year
|
Suppose a forecaster in 1925 used historical data available at that time to predict how
population and the use of streetcar/light rail, rapid rail transit, bus, and automobiles
would change from 1925 to 1950. The figure on the left shows the forecaster's
predictions for the year 1950. The figure on the right shows what actually happened
in 1950. A comparison of the two figures demonstrates that even national-level
predictions may be difficult to get right. For example, the trend line forecast for
automobiles predicted that there would be six times the number of automobiles in
1950 as there was in 1925. But in reality, the 1950 figure was only twice the 1925 level. |
With only the historical base from
1907 to 1925 to draw from, the
1925 forecaster probably would
have predicted rapid growth in three
of the four transportation modes:
bus ridership, automobile ownership,
and rapid rail transit use, all
outpacing population growth. The
forecaster would have expected
population to continue rising but
not as quickly as those three modes.
An astute 1925 forecaster possibly
would have expected streetcar ridership
to drop, given that stakeholders
in the transit industry were becoming
more receptive toward the bus as a new technology, although discerning
the trend of buses taking
market share from streetcars was
more difficult in 1925 than in later
years. Less knowledgeable forecasters
might have thought the drop in
streetcar ridership since 1920 was
merely an aberration.
"Long-range plans should be updated regularly to evolve
as new information becomes available. In that way, we
are not locked onto a rigid conception of the future." |
Taking these five transportation
trends in turn, a perceptive forecaster
in 1925 might have called half
of them accurately. The forecaster
likely would have predicted the
1950 population just about perfectly,
with the past indications of national
population trends being an accurate
predictor of the present.
High marks also would have been
awarded for the prediction of increased
bus ridership, but the accuracy
stems more from chance than
anything else. Although the forecaster
probably could not have foreseen
the Great Depression, the dominance
of bus over trolley transit, or
World War II rationing-all of which
would affect bus ridership-these
factors would have combined to
make the forecaster's estimate of bus
ridership seem respectable. In short,
the historical trend coincidentally
would give a good prediction in this
particular case.
For the automobile, the high
value predicted for 1950—five times
the level in 1925—would come true
eventually—but not until 1975. The
prediction for electric trolley ridership
also might have been in the
right direction, but the 1950 prediction
would have been higher than it
should have been. Finally, the rapid rail transit ridership would have
proven the most difficult to predict.
Increased urbanization and the early
growth of rapid rail transit prior to
1925 might have suggested continued
growth in this industry by 1950;
however, rapid rail actually declined.
In fact, it is difficult to pick any
25-year horizon and be guaranteed
success in predicting all five trends
accurately, using only data available
up to that point in time, with a possible
exception being the period
from 1975 to 2000. This problem is
exacerbated when smaller area forecasts
must be made for counties or census tracts, where it is much easier
to make forecasting errors. Realistically,
of course, more complex forecasting
models can be developed to
keep estimates "in check." The number
of automobiles can be constrained
to a reasonable proportion
of the population, for example, but
predicting shifts such as the rapid
rise in automobile ownership starting
in 1945 is more difficult.
Looking ahead, planning officials may question how other technologies will develop. Will new technologies proliferate in an exponential manner as in the case of wireless phone usage, or will growth be steadier and more linear, comparable to that of
alternative fueled vehicles?
Conclusions
Long-range transportation
plans are necessarily based on
the assumption that historical
data, combined in some cases
with an understanding of the
transportation environment,
can be used to predict the
environment over some planning
horizon, say, a quarter
century. This is probably accurate
for statewide population
totals and may be accurate for
employment and personal
income growth within large
geographical subareas.
Yet historical examples of
changes in behavior, such as
the mode of transportation
chosen by passengers or the number of miles driven, also are affected
by significant technological or
social changes. And it is difficult to
predict key technological and social
developments decades into the future,
such as the innovations in the
rubber-tired bus over the streetcar
during the 1920s, World War II during
the 1940s, the oil embargo of the
1970s, or the rise in personal incomes
in the 1990s. In a similar vein,
it is not yet clear whether technologies,
such as hybrid vehicles, or social
movements, such as telecommuting,
will see the rate of market penetration
deviate from recent trends.
Virginia Population Projections to 2025
From Different Data Sources
|
The figure shows statewide population estimates for Virginia derived from three sources: U.S. Census Bureau (Census), NPA Data Sources, Inc. (NPA), and the Virginia Employment Commission (VEC). In the short term, the estimates converge and are relatively close, falling between 7.0 million and 7.1 million in 2000 and between 7.5 million (VEC) and
7.9 million (NPA) in 2010. By 2025, however, the estimates are further apart: nearly 8.5 million (Census) versus 9.25 million (NPA). The variation in these estimates illustrates some of the uncertainty in making projections. For example, depending on
the forecast, there can be a difference of nearly 1 million people when projecting the State's population 25 years into the future. Source: VTRC. |
Other researchers also have noted the challenges to making predictions. In A Guide to Smart Growth: Shattering Myths, Providing Solutions, Jane Shaw and Ronald Utt, for example, note that in the 1920s, it would have been difficult to predict
80 years later that less than 2 percent of the U.S. population would work in agriculture. In fact, after finding significant differences in model forecasts of transportation and land use impacts, in the article "Comparisons from Sacramento Model
Test Bed" in the Transportation Research Record series, the authors call 25-year forecasting a "bit of a fool's game." They suggest that truly prescient forecasters, if they exist, would invest in real estate speculation rather than urban planning.
The challenges, do, however, suggest
two options to improve long-range
plans. The first is to point out
explicitly that not all trends can be
forecast equally well. Some key
trends, such as population, may be
relatively feasible to predict, whereas
others, such as changes in telecommuting,
are more difficult.
One way to do improved long-range plans is to present estimates with ranges, such as population projections for Virginia from different data sources. The purpose of this approach is to demonstrate the disparity in forecasts from different but
credible sources, as opposed to portraying the "most" accurate forecast. Another alternative to address the
uncertainty, according to Tom Gillaspy of the State Demographic Center in Minnesota, is to perform an analysis using scenarios to examine how changes in key variables will affect a prediction.
"Since the size of the labor force in Minnesota is a function of two factors, future migration rates and future participation rates, we can obtain four different sets of predictions for the labor force in 2030," he says. "They include the combinations of low migration and participation, low migration and high participation, high migration and low participation, and high migration and high participation."
The second option for improving long-range plans is to tie the recommendations explicitly to the confidence that the planner has in the underlying trends. Suppose a multimodal plan suggests targeting resources toward providing greater travel choices because of expected increases in the proportion of the population aged 65 and older. A planner could ask, therefore, how sure are
we that the State will continue to mirror national trends that show an increase in drivers over age 65, and to what extent should we assume that the behavior of this population will be similar to persons in that category today?
One answer is to review relevant literature. In the 1997 report Societal Trends: The Aging Baby Boom and Women's Increased Independence, for example, prepared on behalf of the Federal Highway Administration, Daphne Spain suggests that in 2030 women drivers age 75 and over may drive almost three times as many miles as women in that category at present.
Scenario Planning: A Framework for Developing a Shared Vision for the Future
One answer to more accurate long-range planning might be scenario planning—an analytical tool that can help elected officials, the public, and transportation professionals prepare for what lies ahead. By considering the various factors that will shape the future, scenario planning can help inform and involve the public, ideally to facilitate consensus on how to deal with growth, accommodate future transportation needs, ensure a quality environment, and provide for an aging population.
Scenario planning provides a framework for developing a shared vision for the future by analyzing various forces that affect growth, such as health, transportation, economic development, environment, and land use. Scenario planning, which can be done at the statewide level or for metropolitan areas, tests various future alternatives that meet State and community needs. A defining characteristic of scenario planning is that it actively involves the public, the business community, and elected officials on a broad scale, educating them about growth trends and tradeoffs, and incorporating their values and feedback into future plans.
Scenario planning expands upon traditional planning techniques by focusing on major forces or drivers that have the potential to affect the future. By developing scenarios to tell a story of the future, planners are better able to recognize these forces and determine what planning activities can be done today and can be adapted in the future. Scenario planning is not intended to replace traditional planning practices. It is a process that can be applied to recognize the range of outcomes in the future, beyond what traditional planning can create.
A number of jurisdictions have used scenario planning successfully. To encourage others, FHWA is helping to identify opportunities for the use of scenario planning and providing technical assistance. FHWA is reaching out to FHWA divisions to work with their State DOTs and MPOs to explore specific actions to help improve the planning process using scenario planning tools. For more information, contact Sherry B. Ways, Transportation Planner, FHWA Office of Planning at 202–366–1587 or e-mail sherry.ways@fhwa.dot.gov.
–Sherry B. Ways
|
Neither recommendation is a panacea. Given the desire to make transportation plans more transparent rather than more complex, the decision to add detail to a plan in the form of statements about uncertainty should not be taken lightly. "An important
facet of transportation plans," says Louis Tognacci, senior planner at the Arizona DOT, "is that they distill a
few basic concepts that can be communicated to a wide audience of nonspecialists. Thus, presenting a range instead of a point estimate may add unnecessary complexity."
Ranges, however, represent a feasible starting point for making long-range plans more representative of what is currently understood regarding the future.
"Our long-range planning provides a context that assists in guiding current decisionmaking," Tognacci adds. "Long-range plans should be updated regularly to evolve as new information becomes available. In that way, we are not
locked onto a rigid conception of the future."
John S. Miller, Ph.D., P.E. is a research scientist with VTRC.
References are available in the online version of PUBLIC ROADS. For more information, contact john.miller@virginiadot.org or
access the trends report at http://virginiadot.org/vtrc/main/online_reports/pdf/03-tar5.pdf. For information on Virginia's 2025 plan, see www.sotrans.state.va.us/VTrans/home.htm.
The author thanks J. Gillespie, S. Brich, A. O'Leary, L. Evans, R. Combs, and E. Deasy of VTRC; C. Burnette of the Virginia Department of Aviation; G. Conner and G. Robey of the Virginia Department of Rail and Public Transportation; J. Florin of the Virginia Port Authority; B. Lambert of FHWA; D. Covey, R. Gould, K. Graham, K. Lantz, R. McDonald, K. Spence, D. Wells, and R. Tambellini from VDOT; J. Lambert of the University of Virginia; J. Knapp of the Weldon Cooper Center for Public
Service at the University of Virginia; N. Terleckyj of NPA Data Services, Inc.; and L. Tognacci of Arizona DOT. The inclusion of these names and agencies does not, however, imply agreement with the contents of this article.
Other Articles in this issue:
Taking the High Road
The Space Between
Designing Tomorrow's Pavements
Learning from the 2003 Blackout
Rustic Pavements
I-95 Shutdown—Coordinating Transportation and Emergency Response
Traffic Safety Information Systems
Preventing Corrosion in Steel Bridges
The Uncertainty of Forecasts
Testing Truncated Domes