Planning magazine recently (December 2009) published a story on the benefits of economic impact studies for planners. Most professional planners have run across them at one point or another: they are used to evaluate economic impact and the effectives of various types of programs on job creation. Unfortunately, the article did little to also illuminate the pitfalls and weaknesses of these studies.
This is a particularly timely topic because of the role economic impact studies have played in the national debate over the so-called economic stimulus program. Not surprisingly the focus of the article in Planning was on a particular type of economic impact study, ones that use "multipliers" to estimate the effects of spending on particular projects for a local economy. The approach is used by macroeconomists to estimate national impacts, as well. While other approaches to estimating impacts exist, multiplier analyses are the most common because they are 1) empirical in nature, 2) based on estimates of how dollars flow from one sector of the economy to another, 3) use what are called "input-output" tables that make these relationships transparent (at least to practitioners).
So, if money is spent in the construction sector, the input output table will track how the dollar flows into retail, entertainment, or even purchases of goods within the construction industry. As Consultant Stephen Vickner explains in "When Deep Pockets Open":
Input-output models are quite useful and practical for assessing the economic impact of expansionary government policies as well as private-sector investments. They are perhaps the most popular methodology in a broader toolkit of regional economic analysis methodologies (including multimarket modeling and computable general equilibrium modeling). Planners can readily evaluate the merits of alternative uses of public funds and easily compare the economic impacts of competing project proposals. Moreover, input-output models can be seamlessly integrated with popular planning tools such as GIS analysis to help spatially visualize different measures of economic impact analysis.
Despite Mr. Vickner's enthusiasm, input-output studies suffer from numerous flaws and drawbacks. Planners should be aware of these in order to ensure they are used in the right context. First, input-output studies, as well as any analysis that calculates a "multiplier" to assess impacts, are fundamentally forecasting tools. They are used to predict and estimate future impacts. They do not estimate or measure outcomes. Nor do they evaluate what actually happens.
In this light, Holy Cross economist Victor Matheson's paper on the "before" and "after" studies of economic impact is worth a read because it explores why the real world results of investments often differ significantly from the forecasted impacts. Estimates from input-output studies carry a great deal of uncertainty about real world impacts because they cannot control for the dozens, often hundreds, of other factors that might influence the size and interdependencies of dollars flowing through a regional economy with diverse economic sectors. For example, multipliers estimated for residential construction during the height of the housing boom may be highly inaccurate for trying to forecast impacts in the current economic recession while the industry struggles to stabilize. Indeed, the real-estate market is arguably in an economic depression, and economic shocks of this magnitude have the tendency to radically change all sorts of economic relationships.
Second, multiplier-based studies are not rooted in an analysis of the productivity or efficiency of spending. Economic impact studies (either national or local) do not account for revenues diverted from other parts of the economy, or the opportunity costs (what investment opportunities are foregone) because of those diversions, and how these shifts influence productivity gains downstream. The Texas economy, for example, has weathered the recessionary storm relatively well, while my home state of Ohio continues to struggle with significant employment and investment dislocations. An investment in an automobile plant (or highway) would have different impacts in Texas compared to Ohio. But input-output studies aren't capable of making judgements about spending productivity in fluid and dynamic economic environments. Estimated impacts are based on multipliers derived from past behavior in specific economic sectors. They are not based on industry or economic sectoral studies, and the impacts assume a lump sum spending impact at particular times. They don't and cannot account for the adjustments in the economy that result from those spending injections.
Third, input-output studies don't factor in the effects of debt and borrowing costs to fund projects. Yet, incurring higher debt to finance current projects could compromise long-term growth by limiting future borrowing capacity or necessitating higher taxes to pay off the bonds. Thus, in California, the input-output studies will project thousands of new jobs and higher incomes from funding high-speed rail projects while ignoring the effects of the higher debt incurred to pay for them. Similarly, input-output studies cannot evaluate the productivity of different types of investments, such as a high-occupancy toll lane project in Northern Virginia that reduces travel time and reduces the tax burden of funding other road investments, versus a road project in Pennsylvania that is funded by higher sales taxes.
Despite these criticisms, multiplier analysis should not be dismissed altogether. Multipliers exist in the real world, and the estimates are not fabricated out of thin air. The key is to use the data and analysis in the right context, with the right qualifications, and with more than a little grain of salt.
