Definitions

Definition of ITS Program Assessment/Evaluation

What is ITS Evaluation?

Evaluations are critical to ensuring progress toward the vision of integrated ITS and achieving ITS deployment goals. Evaluations are also critical to an understanding of the value, effectiveness, and impact of National ITS Program activities and allow for the program's continual refinement.

The National ITS Program follows the spirit of the Government Performance and Results Act (GPRA) to help ensure that the program is effective in meeting the U.S. DOT's transportation goals. In keeping with GPRA, tracking of both program outputs and outcomes is emphasized. Program outputs track the progress of a program (e.g., the number of toll plazas equipped with electronic toll collection capability). Program outcomes track the benefits of a program from the perspective of the end-user (e.g., reduction in delay waiting to pay tolls). Another activity is outreach, where evaluation results are communicated to select target audiences in ways that are meaningful to them. In addition, under ITS Evaluation sponsorship, in-depth studies are conducted concerning modeling and simulation of the impact of ITS deployments, estimating the costs and benefits of ITS technologies, determining user acceptance of ITS products and services, and investigating institutional and policy issues related to ITS.

Program Outputs

For the National ITS Program, "output" is defined as the amount of integrated ITS deployed across the nation. The Federal government is leading the tracking of infrastructure deployments, and the U.S. DOT's private sector partners will track the market penetration of components of the Intelligent Vehicle Initiative (IVI).

Metropolitan

The metropolitan ITS deployment goal – "Operation TimeSaver" – is to deploy an integrated ITS infrastructure in the nation's 75 largest metropolitan areas by 2006. The ITS Deployment Tracking Database contains the results of surveys of metropolitan areas to measure progress towards this goal. The database contains the results of surveys of metropolitan areas taken in FY96, FY97, FY99, FY00 and FY02 regarding how much ITS equipment they have actually deployed. Users can view survey results by component and by metropolitan area, view blank surveys, and review the precise definitions used to determine how much ITS is deployed in each metropolitan area. Users can also compare one metropolitan area to another, as well as to the national average.

Commercial Vehicle Operations

The commercial vehicle operations deployment goal is to deploy initial operating systems and Commercial Vehicle Information Systems and Networks (CVISN) capabilities in a majority of states by 2003. The CVISN section of the ITS Deployment Tracking Database contains the results of surveys of states to measure progress towards this goal. The database contains the results of surveys of states taken in FY96 and FY98 regarding their CVISN capabilities. Users can view survey results by component and by state, view blank views, and review the precise definitions used to determine how much ITS is deployed in each state. Users can also compare one state area to another, as well as to the national average.

Rural and Statewide

Currently there is no deployment goal for rural and statewide ITS systems. However, in FY02 several surveys were conducted to measure the level of rural and statewide ITS in select locations. The ITS Deployment Tracking Database contains the results of surveys of 50 states, 30 medium-sized cities, and 20 tourist cities regarding their ITS deployment. Users can view survey results by component or by city or state, view blank surveys, and review the precise definitions used to determine how much ITS is deployed in each city or state. For the medium-sized cities and tourist cities, users can compare one city to another, as well as to the average of this type of city surveyed.

Program Outcomes

Program outcomes are measured according to a few key measures of program effectiveness derived from the 1992 document Strategic Plan for Intelligent Vehicle Highway Systems in the United States. These measures apply across all infrastructure and IVI areas of the program. The table below relates ITS strategic goal areas with key outcome measures:

It would be impractical at the Federal level to attempt to measure the impact of all ITS deployments in every possible metropolitan area, state, region and vehicle platform. However, it is practical to compile a database of known impacts of specific ITS technologies and use that information to make sound predictions of the impact of future ITS deployments.

The ITS Benefits and Costs Database contains information regarding the impact of ITS projects on the operation of the surface transportation network (the ITS benefits database) and estimates of ITS costs data that can be used for policy analysis and benefit/cost analysis (the ITS unit costs database). Entries in the ITS benefits database are brief summaries of ITS benefits reported in printed and online sources. At present, the ITS benefits database contains almost 200 entries. Reference information is provided for each entry, along with a link to the reference document if available electronically. Users can search the ITS benefits database by ITS component, performance measure, location and date submitted. The ITS costs database contains unit cost estimates for over 200 ITS technologies. Users can view the database online, as an Excel spreadsheet, and as a four-page Adobe Acrobat document. Users are invited to contribute new information to both the ITS benefits and costs databases, and the website has instructions on how to do so.

To measure the impacts of metropolitan, rural and statewide, and commercial vehicle ITS technologies, conclusions are drawn from three primary sources: (1) operational tests with evaluations sponsored by the U.S. DOT; (2) locally-sponsored ITS evaluations and deployments; and (3) the Metropolitan and CVISN model deployment programs. To measure the impacts of IVI technologies, conclusions are drawn primarily from U.S. DOT-sponsored research studies. Source documents are available in the Documents section of this website.

Program Outreach

Outreach for the ITS Program Assessment and Evaluation program involves communication of ITS evaluation results to target audiences in ways that are meaningful to them. The primary method of outreach employed to date has been publication of colorful eye-catching documents, the purpose of which is to educate state and local decision-makers, mid-level managers, and technical staff about the benefits of ITS, what works, what doesn't, and how to implement ITS. Five types of documents have been developed, each with its own format designed to communicate to a specific target audience: brochures, cross-cutting studies, case studies, implementation guides, and technical reports. These documents are available in the Documents section of this website and are identified as "ITS Benefits Special Studies". Additional outreach activities include hosting evaluation-related sessions and presentations at professional conferences, and development of innovative communications media such as videos and possibly interactive CD-ROMs.