PROGRAM EVALUATION

Program evaluation is one of the three major elements of the Government Performance and Results Act (GPRA). The statute calls for agencies to use program evaluations to assess the manner and extent to which Federal programs achieve intended objectives. The statute further calls for an agency's Performance Plan to include a summary of the findings of program evaluations completed in the fiscal year covered in the report. Finally, GPRA calls for a schedule for future program evaluations to be presented in Strategic Plans.

In response to these mandates, DOT reports its completed program evaluations annually in “The Department of Transportation Performance and Accountability Reports.”[8] To the extent that the results of completed program evaluations, Program Assessment Rating Tool (PART) reviews, and the reports of the DOT Office of the Inspector General and the General Accountability Office illuminate how we might achieve results more effectively or address future conditions, we considered these in writing the strategies presented in this Strategic Plan.

Table 7 below presents the schedule for future DOT program evaluations. These evaluations represent a cross-section of DOT programs that must be well-managed in full support of budget-performance integration. DOT defines the methodologies used in Table 7 program evaluations as follows.

Impact Evaluations use empirical data to compare measurable program outcomes with what would have happened in the absence of the program. These represent the highest standard of program evaluations and are often the most difficult and expensive to construct and interpret.

Outcome Evaluations assess the extent to which programs achieve their outcome-oriented objectives. Outcome evaluations use quantitative methods to assess program effectiveness.

Process Evaluations assess the extent to which a program is operating as intended. While a true process evaluation will use objective measurement and analysis, it does not address the causal links between intervention and outcome.

Cost-Benefit and Cost-Effectiveness Analyses compare a program's outputs or outcomes with the costs to produce them. This type of analysis conforms to program evaluation when applied systematically to existing programs and when measurable outputs and outcomes are monetized.

Table 7. Program Evaluations for Fiscal Years 2006-2011

Agency	Strategic Goals[9]					O E	Methodology	Title	FY Completed
Agency	S	R	G	E	P	O E	Methodology	Title	FY Completed
RITA						X	Process - NRC Review and GAO Study	RD&T Strategic Plan	2006
FMCSA	X						Outcome - conducted by an independent auditor	Commercial Driver's License Program	2006
FMCSA	X	X	X		X	X	Process - conducted by an independent auditor	Performance Linkages	2006
FAA					X		Outcome	Facility Security Program	2006
PHMSA	X			X			Outcome	Retrospective assessment of benefits and impacts of the pipeline safety operator qualification regulations.	2007
PHMSA	X			X		X	Process - conducted by an independent auditor	Assess the current information technology (IT) program to identify overlapping and redundant IT investments, systems and services.	2007
FAA	X		X				Outcome	Safer Skies	2007
MARAD					X		Impact assessment - conducted by an independent auditor	Maritime Security Program	2007
FAA	X						Outcome	Operational Error Program	2008
NHTSA	X						Outcome - conducted by an independent auditor	Evaluation of National Mobilizations	2006 2007 2008
FMCSA	X						Outcome - conducted by an independent auditor	Compliance Review (CR) Effectiveness	Annually 2006 through 2011
FMCSA	X						Outcome - conducted by an independent auditor	Roadside Inspection and Traffic Enforcement Effectiveness	Annually 2006 through 2011
FRA	X	X		X	X		Process - conducted by an independent auditor	Review of FRA's Research, Development and Demonstration Programs	Annually 2006 through 2011
FTA/FHWA	X	X	X		X		Cost Effectiveness	Infrastructure Investment Needs Report (Conditions and Performance Report)	Biennially 2006 through 2011
FRA	X					X	Outcome - conducted by an independent auditor	Railroad Safety Enforcement	2006 2007
NHTSA	X						Outcome	Side Impact Protection and Side Air Bags	2007
FMCSA	X						Process - conducted by an independent auditor	SafeStat Program	2007
FMCSA	X						Process - conducted by an independent auditor	Alternative Inspection Regimes	2007
FTA		X		X			Cost Benefit	Public/Private Partnership Program	2007
PHMSA	X			X	X		Process	Evaluation of outreach and training programs	2008
FMCSA	X				X	X	Process/cost effectiveness - conducted by an independent auditor	State Division Effectiveness	2008
FMCSA	X				X		Process/cost effectiveness - conducted by an independent auditor	Motor Carrier Safety Assistance Program (MCSAP)	2008
FHWA/FTA		X					Cost effectiveness	Infrastructure Investment Needs Report	2008
FAA		X					Process	Aircraft Delay Reduction Program	2008
FMCSA	X						Process – conducted by an independent auditor	Quality Assurance Review – Grants Management	2008
FTA		X					Outcome	Job Access and Reverse Commute Formula Grant Program	2008
MARAD					X		Impact assessment - conducted by an independent auditor	Maritime Education Programs	2008
PHMSA	X			X			Outcome Process	Evaluation of the pipeline safety State grants program	2009
PHMSA	X						Process	Evaluation of processing pipeline and hazmat enforcement cases	2009
PHMSA	X						Process	Evaluation of readiness among emergency responders	2009
RITA						X	Process - conducted by an independent auditor	UTC Program	2009
FAA				X			Outcome	Strategic Sourcing	2009
FMCSA	X						Outcome – conducted by an independent auditor	New Entrant Safety Audits	2009
FMCSA	X						Cost effectiveness – conducted by an independent auditor	Information Management Program	2009
FTA		X					Outcome	Contracted Para-transit Pilot Program	2009
MARAD				X			Process – conducted by an independent auditor	Ship Disposal	2009
FMCSA	X						Outcome - conducted by an independent auditor	Motor Coach Operations	2010
RITA		X	X				Cost Benefit	BTS – Airline Program Web Filing	2010
FMCSA	X						Process	Quality Assurance Review – Enforcement Activities	2010
FHWA	X	X	X	X	X		Outcome	Strategic Highway Research Program II—Report on Implementation of Results	2010
FHWA	X	X	X	X	X		Process	Strategic Highway Research Program II—Programmatic Evaluation	2010
FAA				X			Process	Streamlined Environmental Impact Statement Process	2010
MARAD			X				Outcome - conducted by an independent auditor	Cargo Preference – Food Aid Cargoes	2010
PHMSA	X			X			Impact	Effectiveness of the pipeline safety integrity management program	2011
FAA	X						Outcome	Runway Safety Program	2011

Strategic Plan Table of Contents | Previous | Next