Summary
Using performance measures, the Coast Guard explains how well its programs are performing. To do so, it reports one "primary" measure for each program (such as percent of mariners rescued) and maintains data on other, "secondary" measures (such as percent of property saved). Concerns have been raised about whether measures for non-homeland security programs accurately reflect performance, that is, they did not rise or fall as resources were added or reduced. For the six non-homeland security programs, GAO used established criteria to assess the soundness of the primary measures--that is, whether measures cover key activities; are clearly stated; and are objective, measurable, and quantifiable--and the reliability of data used to calculate them. GAO also used these criteria to assess the soundness of 23 selected secondary measures. Finally, through interviews and report review, GAO assessed challenges in using measures to link resources to results.
While some opportunities for improvement exist, the primary measures for the Coast Guard's six non-homeland security programs are generally sound, and the data used to calculate them are generally reliable. All six measures cover key program activities and are objective, measurable, and quantifiable, but three are not completely clear--that is, they do not consistently provide clear and specific descriptions of the data, events, or geographic areas they include. Also, the processes used to enter and review the Coast Guard's own internal data are likely to produce reliable data; however, neither the Department of Homeland Security (DHS) nor the Coast Guard have policies or procedures for reviewing or verifying data from external sources, such as other federal agencies. Currently, the review processes vary from source to source, and for the primary measure covering marine environmental protection (which concerns oil and chemical spills), the processes are insufficient. Of the 23 secondary performance measures GAO assessed, 9 are generally sound, with weaknesses existing in the remaining 14. These weaknesses include (1) a lack of measurable performance targets, (2) a lack of agencywide criteria or guidance to ensure objectivity, and (3) unclear descriptions of the measures. Two main challenges exist with using primary measures to link resources to results. In one case, the challenge is comprehensiveness--that is, although each primary measure captures a major segment of program activity, no one measure captures all program activities and thereby accounts for all program resources. The other challenge involves external factors, some of which are outside the Coast Guard's control, that affect performance. For example, weather conditions can affect the amount of ice that must be cleared, the number of aids to navigation that need repair, or mariners that must be rescued. As a result, linking resources and results is difficult, and although the Coast Guard has a range of ongoing initiatives to do so, it is still too early to assess the agency's ability to successfully provide this link.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
Stephen L. Caldwell
Government Accountability Office: Homeland Security and Justice
(202) 512-9610
Recommendations for Executive Action
Recommendation: To improve the quality of program performance reporting and to more efficiently and effectively assess progress toward achieving the goals or objectives stated in agency plans, the Secretary of Homeland Security should direct the Commandant of the Coast Guard to refine certain Coast Guard primary and secondary performance measures by further clarifying the ice operations primary measure by clearly and consistently describing the geographic area and number of waterways included in the measure; the living marine resources primary measure by clearly and consistently reporting the scope of the measure; and the search and rescue primary measure and the search and rescue "percent of lives saved after Coast Guard notification" secondary measure by reporting those incidents or data that are not included in the measures.
Agency Affected: Department of Homeland Security: United States Coast Guard
Status: In process
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Recommendation: To improve the quality of program performance reporting and to more efficiently and effectively assess progress toward achieving the goals or objectives stated in agency plans, the Secretary of Homeland Security should direct the Commandant of the Coast Guard to refine certain Coast Guard primary and secondary performance measures by developing measurable performance targets to facilitate assessments of whether program and agency goals and objectives are being achieved for the 11 living marine resources secondary measures and the 1 marine environmental protection secondary measure, "Tokyo and Paris memorandums of understanding port state control reports," that lack annual targets.
Agency Affected: Department of Homeland Security: United States Coast Guard
Status: In process
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Recommendation: To improve the quality of program performance reporting and to more efficiently and effectively assess progress toward achieving the goals or objectives stated in agency plans, the Secretary of Homeland Security should direct the Commandant of the Coast Guard to refine certain Coast Guard primary and secondary performance measures by establishing agencywide criteria or guidance to help ensure the objectivity and consistency of the search and rescue program's "percent of property saved" secondary performance measure.
Agency Affected: Department of Homeland Security: United States Coast Guard
Status: In process
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Recommendation: To improve the quality of program performance reporting and to more efficiently and effectively assess progress toward achieving the goals or objectives stated in agency plans, the Secretary of Homeland Security should direct the Commandant of the Coast Guard to develop and implement a policy to review external data provided by third parties that is used in calculating performance measures to, at a minimum, be familiar with the internal controls external parties use to determine the reliability of their data.
Agency Affected: Department of Homeland Security: United States Coast Guard
Status: In process
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
Recommendation: To improve the quality of program performance reporting and to more efficiently and effectively assess progress toward achieving the goals or objectives stated in agency plans, the Secretary of Homeland Security should direct the Commandant of the Coast Guard to report additional information--besides the one primary measure--in appropriate agency publications or documents where doing so would help provide greater context or perspective on the relationship between resources expended and program results achieved.
Agency Affected: Department of Homeland Security: United States Coast Guard
Status: In process
Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.
