Detailed Information on the
Coast Guard: Marine Safety Assessment

Develop a plan for regular, independent assessments of its programs' performance. Coast Guard has a schedule to conduct one-time evaluations of all of its programs, but it needs to develop a more routine and regular process for evaluation.

Ensure MISLE MS data is credible and reliable.

Create annual measure scorecard. The Coast Guard should work to institutionalize supporting operational measures that help program managers better understand factors that contribute to annual program performance achievement. Concurrent with these measures, the Coast Guard should set ambitious performance targets to encourage performance improvements. These measures should be promulgated to all program managers on a routine basis.

Harmonize recreational boating regulations. The Coast Guard should work with its recreational boating partners (Department of the Interior, Army Corps of Engineers) to harmonize overlapping federal regulatory structures and standards.

Expand MS measures.

Measure: Five-year average number of commercial mariner deaths and injuries

Explanation:This is a measure of the long-term performance trend of the U.S. Coast Guard Marine Safety Program impact on commercial mariner fatalities and injuries.

Measure: Five-year average number of commercial passenger deaths and injuries

Explanation:This is a measure of the long-term performance trend of the U.S. Coast Guard Marine Safety Program impact on commercial passenger fatalities and injuries.

Measure: Five-year average number of recreational boating deaths and injuries

Explanation:This is a measure of the long-term performance trend of the U.S. Coast Guard Marine Safety Program impact on recreational boating fatalities and injuries.

Measure: Commercial mariner deaths and injuries

Explanation:This measure is an indicator of U.S. Coast Guard Marine Safety Program impact on the annual number of commerical mariner fatalities and injuries.

Measure: Commercial passenger deaths and injuries

Explanation:This measure is an indicator of U.S. Coast Guard Marine Safety Program impact on the annual number of passenger fatalities and injuries.

Measure: Recreational boating deaths and injuries

Explanation:This measure is an indicator of U.S. Coast Guard Marine Safety Program impact on the annual number of recreational boating fatalities and injuries.

Measure: Marine Safety Efficiency Ratio

Explanation:This is a measure of the relative change in long-term Marine Safety Program performance versus relative change in long-term cost. A value greater than 1.000 indicates performance efficiencyperformance gains greater than cost changes. For example, a value of 1.050 indicates a 5% improvement in program efficiency.

Is the program purpose clear?

Explanation: The purpose of the Coast Guard's Marine Safety Program is to prevent deaths, injuries, and property losses in the maritime domain. The Marine Safety program is comprised of two principal components - Commercial Vessel Safety (CVS), and Recreational Boating Safety (RBS). The CVS component includes the Coast Guard's responsibilities for developing federal marine safety regulations, admeasurement and documentation of U.S. vessels, management of the U.S. Merchant Mariner Licensing and Documentation program (certifying the competency and qualifications of over 200,000 mariners), and supporting a variety of government and non-government recreational boating safety efforts. The Coast Guard also represents the United States at the International Maritime Organization, and contributes in the development of international vessel and mariner standards such as the Convention for Safety of Life at Sea (SOLAS), and the International Convention on Standards for Training, Certification and Watchkeeping for Seafarers. The Coast Guard conducts compulsory inspections??as well as voluntary examinations??and enhances marine safety through incentive programs and in partnership with other federal agencies, state and local governments, marine industries and associations. The Coast Guard also promotes safe practices by investigating marine casualties and disseminating its findings through the Proceedings of the Marine Safety Council, Navigation & Vessel Inspection Circulars, and other policy and guidance. For the RBS component, the Coast Guard manages a comprehensive boating safety programs, including a grant program to encourage the States to participate, and promulgation of vessel construction and performance standards. The Coast Guard has developed manufacturer regulations in the areas of floatation, capacity, fuel systems, electrical systems and ventilation systems. Additionally, the Coast Guard has promulgated safety equipment carriage requirements for recreational boaters.

Evidence: ?? USCG Publication 1 ?? Coast Guard 2020 ?? USCG Strategic Blueprint ?? United States Code, Title 46 (relevant sections only) ?? Public Law 107-296 Homeland Security Act of 2002 (http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=107_cong_public_laws&docid=f:publ296.107.pdf) ?? H.R. 5005 Homeland Security Act of 2002 (relevant section only) ?? Recreational Vessel Regulations - 33 CFR 173, 174, 175, 179, 181, 183

Does the program address a specific and existing problem, interest, or need?

Explanation: Despite Coast Guard's best efforts, recreational boaters, maritime industry workers, and commercial vessel passengers still perish each year, and marine safety remains an expressed public concern. Recent commercial multiple-death casualties include the Staten Island Ferry allision (11 fatalities), the LADY D pontoon boat capsizing and sinking in Baltimore Harbor (5 fatalities), and the allision of barges directed by the tug ROBERT Y. LOVE into the I-40 bridge in Oklahoma causing its collapse(14 fatalities). All these casualties occurred due to the failure of the vessel operator or managers to ensure compliance with required vessel safety and operations standards. In addition, there were 710 recreational boating fatalities in calendar year 2003. Recreational boating remains the mode of transportation that results in the second highest number of annual fatalities, next to automobiles, and it remains on the National Transportation Safety Board's Most Wanted List.

Evidence: ?? Public Law 108-293 Coast Guard and Maritime Transportation Act of 2004 ?? Conference Report on H.R. 2443 Coast Guard and Maritime Transportation Act of 2004, 21 July 2004 ?? Public Law 107-296 Homeland Security Act of 2002 (relevant text at frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=107_cong_public_laws&docid=f:publ296.107.pdf) ?? H.R. 5005 Homeland Security Act of 2002 (relevant section only) ?? USA Today: 400 Stricken Cruise Ship Passengers, 7 Jun 2004 ?? The Virginia Pilot: Crew Rescued off Virginia Coast, 2000 ?? Annual Boating Statistics 1997-2003 (www.uscgboating.org/statistics/accident_stats.htm)

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Coast Guard has specific statutory authority to develop and enforce marine safety standards throughout the country. Although there are several examples of duplicative programs and organizations that focus on both commercial and recreational boat safety, the bulk of the Coast Guard Marine Safety program is not redundant, and the Coast Guard has taken steps to ensure that the design of the program does not cause unnecessary duplication. In the Commercial Vessel Safety realm, the majority of US-flagged vessels are inspected by the Coast Guard. A small population of vessels qualify for inspection by private organizations called Classification Societies. These inspections affect 450 eligible larger commercial vessels and some towing vessels, and are administered through the Alternative Compliance Program (ACP). The Coast Guard has minimized duplication by designing the ACP so that the private inspectors evaluate vessels for safety standards as developed by the Coast Guard, while the Coast Guard uses a rigorous certification process oversee the participating Classification Societies. A portion of the Recreational Boating Safety program -- namely the federal recreational boating operating regulations -- overlaps with other Federal programs that also issue operator regulations. For example, the National Park Service, U.S. Army Corps of Engineers, and Tennessee Valley Authority all have marine safety regulatory authority for their specific sites or facilities. However, the Coast Guard is making efforts to harmonize federal recreational boating regulations, and the other elements of the Recreational Boating Safety program do not have overlap with other efforts.

Evidence: ?? CG Marine Safety Manual, Volume X: ?? CG Marine Safety Manual, Volume II, Section B, Chapters 9 and 10 ?? 46 CFR Part 8, Alternate Compliance Programs ?? 33 CFR Part 96, Safety Management Systems ?? NVIC 2-95, CH-1; ACP ?? NVIC 2-99, SIP ?? MOA Between the Commonwealth of Virginia and the USCG (This is an example MOU, representative of the MOUs that are currently in effect with all 56 States and Territories.)

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The regulatory component of the Coast Guard Marine Safety Program incorporates cost analysis into its notices of public rulemaking to ensure the program is maximizing net benefits. The program also identified and addresses emerging challenges through continuous program measurement, marine casualty investigation findings, active consultation with industry and other partners, and comparisons to other International Maritime Organization (IMO) members to assess its effectiveness and efficiency. As opportunities for improvement are uncovered, policy and practices are adjusted. For the Recreational Boating Safety component, regulations and administrative processes are kept to the minimum required to oversee the program. Working through State partners provides local boating safety services with minimum Federal cost and intervention.

Evidence: ?? Port State Control Program Directives ?? Fishing Vessel Safety Task Force "Dying to Fish" - 1999 ?? Cruise Ship Safety Task Force Study - 1995 ?? Towing Vessel Safety Study 2002 ?? Quality Standards Systems (QSS) for MLD Program ?? Annual Harbor Safety Committee Conferences

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Coast Guard uses a risk-based decision making approach to direct its efforts and resources toward the most pressing needs; uses outcome measures to monitor its efforts and ensure they address program purposes; and incident investigation & analysis processes to ensure that Coast Guard policies and practices address the needs of intended beneficiaries. In addition to primary outcome data consisting of deaths, injuries, property damage, and vessel sinkings; we also use secondary data. For example, the Port State Control Program tracks deficiencies by ship type, history, class, flag, and owner; publishes the results annually; and uses this data to set boarding priorities. These indicators provide warning of excessively poor performance by a Flag, owner/operator, company, ship class, or classification society by raising the risk matrix sums for the categories, and placing the poor performing vessel in a status where it will be boarded frequently. Concerning recreational vessels, the Coast Guard's National Boating Safety Advisory Council meets twice per year and provides essential guidance to the Program. Also, the Coast Guard recently began a strategic planning process with its partners, developing new, collective performance goals; and they will soon be developing related objectives and strategies. (Objectives to be discussed and drafted at a meeting prior to November 2005.)

Evidence: ?? Port State Control Program ?? Port State Control Annual Report (1998-2004): www.uscg.mil/hq/g-m/pscweb/Publication.htm ?? Fishing Vessel Safety Division (G-MOC-3) homepage: www.uscg.mil/hq/g-m/cfvs/index.htm ?? Outline of Quality Partnership between USCG and American Waterways Operators ?? Annual Boating Statistics 1997-2003 www.uscgboating.org/statistics/accident_stats.htm ?? MOA Between the Commonwealth of Virginia and the USCG National Boating Safety Advisory Council Charter ?? National Boating Safety Advisory Council Minutes, October 2004 meeting

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Marine Safety program has one primary performance metric that incorporates long-term outcome measures specific to three core constituencies: deaths of recreational boaters, deaths & injuries of commercial passengers, and deaths & injuries of commercial mariners. Program managers use these outcome indicators to track marine safety performance. Although this measure is not normalized to account for changes in the population of boaters, the Coast Guard is investigating ways to improve the metric. The Coast Guard is currently working with the States to enhance their reporting of casualty data. As these enhancements take effect, the Coast Guard will be able to provide more timely recreational boating data on both a fiscal and calendar year basis. The Coast Guard will also be able to determine a comparable five-year average that will include recreational boating injuries (similar to the average used with commercial mariners and passengers). This will provide more accurate and comprehensive data for effective analysis and response purposes.

Evidence: ?? USCG Annual Performance Report & Budget In Brief FY 2005 ?? USCG Annual Performance Report & Budget In Brief FY 2004 ?? USCG Quarterly Performance Reports - DHS FYHSP (Not in Binder) ?? USCG Strategic Blueprint ?? USCG Business Plan - Marine Safety ?? Annual Boating Statistics 1997-2003 www.uscgboating.org/statistics/accident_stats.htm ?? Graph of Recreational Boating Fatalities 1962 - 2003 ?? National Boating Safety Advisory Council Goals Resolution ?? DHS Performance and Accountability Report, Fiscal Year 2004 (Not in Binder)

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Although the Coast Guard has a well-established process to monitor marine casualties and deaths, it has not quantified any ambitious long-term goals for the program. Specifically, the Coast Guard has published program performance targets that exceed prior years' achievements. While the Coast Guard claims it has improved the process by which it sets long-term targets, it remains to be seen whether such changes have been permanently implemented. The Coast Guard target-setting process is also of questionable rigor. The Coast Guard claims it sets targets that are "challenging, but also well-grounded in realistic expectations," but there is no evidence of a formal or quantitative methodology for setting long-term goals for the program. One factor that paritally mitigates an apparent lack of ambitious targets for the program's long-term outcome measure is the fact that the boating population, both commercial and recreational, has grown significantly over the past 25 years. Although anecdotal evidence illustrates that the rate of Maritime Casualities normalized for population growth shows more dramatic goal-setting by the Coast Guard, there is no clear and objective evidence to support this claim.

Evidence: ?? USCG Annual Performance Report & Budget In Brief FY 2005 ?? USCG Annual Performance Report & Budget In Brief FY 2004 ?? USCG Quarterly Performance Reports - DHS FYHSP ?? Annual Boating Statistics 1997-2003 www.uscgboating.org/statistics/accident_stats.htm ?? Chart of Annual Number of Mariner and Passenger Deaths & Injuries 1999-2010 ?? Commercial Vessel Quarterly Performance FY05 Q1 & FY05 Q2 (Recreational Boating 2004 information not yet available) as reported for GPRA and FYHSP

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The Coast Guard publishes the same measure to track annual performance as it does long-term performance, and uses a variety of "second-tier" measures to conduct ad hoc analysis of deviation from annual performance targets. While this is not problematic in and of itself, there is no evidence that the Coast Guard has identified or systematically monitors the key drivers of the Marine Safety program's performance, even though this program is a long-running Coast Guard responsibility. As a comparative example, the National Highway Traffic Safety Administration (NHTSA) has recently made progress in identifying leading performance indicators for highway safety that enable managers to better understand performance factors contributing to achievement of overall program goals. The Coast Guard has developed an efficiency measure for the Marine Safety program, and is in the process of implementing its use throughout the Agency. Appropriate annual measures are also under development for the recreational boating program, in consultation with State and local partners of the program.

Evidence: ?? USCG Annual Performance Report & Budget In Brief FY 2005 ?? USCG Annual Performance Report & Budget In Brief FY 2004 ?? USCG Quarterly Performance Reports - DHS FYHSP ?? Annual Boating Statistics 1997-2003 www.uscgboating.org/statistics/accident_stats.htm ?? National Boating Safety Advisory Council Goals Resolution

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Since the Coast Guard's annual measure is the same as its long-term measure, the annual performance targets suffer from the same shortcomings as the long-term targets. Namely, there is little evidence that the targets set by the Coast Guard are ambitious. Specifically, in the most recent Coast Guard Annual Performance Report, annual performance targets for FY 2004 and FY 2005 were above the level of performance achieved in FY 2003. As the Coast Guard further develops a systematic and rigorous approach to operational performance measures used by program managers, it will also have to develop baseline and targets for those new metrics. Similar to the commercial vessel safety component, once specific measures are established with States and local partners for the recreational boating program, program managers will have to set targets and goals for those metrics.

Evidence: ?? USCG Annual Performance Report & Budget In Brief FY 2005 ?? USCG Annual Performance Report & Budget In Brief FY 2004 ?? USCG Quarterly Performance Reports - DHS FYHSP ?? USCG Marine Safety Performance Workbook ?? Annual Boating Statistics 1997-2003 www.uscgboating.org/statistics/accident_stats.htm ?? Graph of Recreational Boating Fatalities 1962 - 2003 ?? National PFD Wear Rate Observational Study - 2003 ?? NBSAC Increased PFD Wear Resolution ?? Chart of Annual Number of Mariner and Passenger Deaths & Injuries 1999-2010

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The Coast Guard's partners in the marine safety realm are all focused on improving safety aboard ships and boats, hence they implicitly commit to work toward the goals of the program. Various partners such as the American Waterways Operators (AWO), Passenger Vessel Association (PVA), International Council of Cruise Lines (ICCL), the National Marine Manufacturers Association, and the American Boat and Yacht Council; provide industry specific input to regulatory regimes. Other groups, such as the Baltic and International Maritime Council (BIMCO) and the International Association of Independent Tanker Owners Association (INTERTANKO) provide technical and diplomatic support to U.S. proposed safety standards for adoption into the instruments of international treaties. Over the years, these partnerships have spawned numerous programs which support regulatory processes and safety initiatives. On the recreational boating front, the Coast Guard's National Boating Safety Advisory Council meets twice per year and provides essential guidance to the Program.

Evidence: ?? BIMCO - Baltic and International Maritime Council; www.uscg.mil/hq/g%2Dm/nmc/ptp/bimco.htm ?? INTERTANKO - The International Association of Independent Tanker Owners Association: www.uscg.mil/hq/g%2Dm/nmc/ptp/intertnk.pdf ?? MOA Between the Commonwealth of Virginia and the USCG ?? MOU Between USCG and the National Association of State Boating Law Administrators

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: No comprehensive evaluation of the Marine Safety program has been conducted to date, so it is impossible to validate conclusions about program effectiveness. Although the Coast Guard plans to contract with the Center for Naval Analyses to review the Marine Safety program, that evaluation will not be completed until 2009 at the earliest. In the past, elements of the Coast Guard Marine Safety program has received third-party scrutiny by GAO and DHS IG auditors, industry partners, advisory groups, and organizations, international organizations and peer states. For example, Deloitte & Touche audited the Merchant Mariner Licensing and Documentation (MLD) Program in 2004, the Paris and Tokyo MOU organizations publish comparative performance statistics for member countries, and the DHS Inspector General audits Coast Guard mission performance on an annual basis. However, these analyses either do not encompass the whole of the Marine Safety program, as in the case of the Deloitte & Touche study, or the materials do not provide any managerial findings specific to the performance of the Coast Guard's program, as with the MOU and DHS Inspector General reports.

Evidence: ?? Internal CG memorandum dated 24 February 2005 regarding out year PART Review and Independent Eval Plan ?? Evaluation of the USCG Quality Standards System (QSS) for the MLD Program by Deloitte & Touche, 2004 ?? Paris MOU Annual Report: www.parismou.org/ ?? Tokyo MOU Annual Report: www.tokyo-mou.org/ ?? DHS IG Audit OIG-04-43 FY2003 Mission Performance of USCG ?? GAO-04-432 USCG Relationship between Resources Used and Results Achieved Needs to Be Clearer

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Coast Guard uses a performance-based budgeting system to tie funding directly to programs and their performance goals and targets. The system used to develop these estimates is in place throughout the budget build process, including resource proposal development, prioritization, and justification. By separating programs' funding levels, budget managers can identify how specific initiatives will generate performance impacts. The Coast Guard's Office of Planning and Performance and Office of Budget and Programs work together to identify and prioritize funding requirements to meet the Coast Guard's mission-program performance goals. For example, to meet the Coast Guard's Marine Safety fiscal year 2006 performance goal of reducing the 5-year average of annual fatalities and injuries to 1,280, the Coast Guard has budgeted $389.7 million within its Operating Expense appropriation to conduct maritime safety and prevention activities. Within this funding level, the fiscal year 2006 budget proposes to increase port presence in Cove Point, MD and Providence, RI to better manage the maritime safety and security risks associated with shipment of liquefied natural gas to nearby facilities.

Evidence: ?? USCG Annual Performance Report & Budget In Brief FY 2005 ?? USCG Annual Performance Report & Budget In Brief FY 2004 ?? Budget Estimates: Fiscal Year 2006.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Coast Guard conducts a robust annual strategic planning review that identifies high-risk/high-return initiatives requiring the attention of executives and managers. Several recommendations from the 2004 Strategic Assessment have already resulted in changes to the Marine Safety program. In addition, the Coast Guard works to identify opportunities for improvement through routine management of its programs. Within the Recreational Boating Safety component of the Marine Safety program, the Office of Boating Safety is developing a Strategic Plan and Risk-Based Management System. Furthermore, a panel was convened to assist in developing additional performance-based measures.

Evidence: ?? 2004 Maritime Strategic Risk Assessment ?? RBS Goals Panel Meeting Minutes

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: The Coast Guard has a well-defined procedure for its regulatory development process. Government-wide standards for development of regulations are supplemented by internal Coast Guard policies to ensure consistency with previously-issued Coast Guard regulations. The Coast Guard also reviews its regulatory plan on an annual basis, based on program goals and priorities, along with input from industry, international activities at the IMO, and other federal advisory committees. This process ensures alignment of regulatory development with program goals. When developing new regulations, the Coast Guard interacts with public and industry stakeholders to ensure regulation efficacy and understanding. For example, the Coast Guard has begun the process of establishing regulations for the inspection of over 5200 towing vessels, as mandated in the Coast Guard and Maritime Transportation Act of 2004. In four public meetings and at Towing Safety Advisory Committee meetings, the Coast Guard has publicly committed to only develop regulatory requirements that address the accident history of the Towing Vessel Industry. The Recreational Boating Safety program has promulgated minimal safety regulations that are properly justified in accordance with its regulatory authority. In addition, the Secretary-appointed National Boating Safety Advisory Council conducts semi-annual reviews of the performance of the recreational boating program and five year reviews of the regulatory framework. This helps assure the regulations for recreational boating safety are appropriate.

Evidence: ?? Coast Guard Headquarters Instruction M16703.1, Regulatory Development and Review, dated January 11, 1995, as modified by M16703.2, dated June 25, 2001. ?? Examples of workplans - Commercial Fishing Vessel Safety, Maritime Security ?? Regulatory Analysis examples - Draft Regulatory Analysis for NPRM, Marine Casualties and Investigations, Chemical Testing Following Serious Marine Incidents, USCG 2001-8773, Regulatory Evaluation for the Final Rule, Towing Vessel Safety: Fire-Suppression Systems and Voyage Planning for Towing Vessels, USCG 2000-6931 ?? Oil Pollution Act of 1990; proposed rules for salvage and fire fighting and dispersants for oil spills http://dms.dot.gov, docket # 3417 ?? Maritime Transportation Security Act; Final Rules; http://dms.dot.gov, docket numbers, 14792, 14733, 14749, 14732,14759, 14757 ?? National Invasive Species Act of 1996; Final rules on penalties for non-reporting and mandatory ballast water management http://dms.dot.gov, docket numbers 13147, 14273 ?? Standards for Training, Certification and Watchkeeping for Seafarers, (CG 95-062, Interim Rule June 1997). The Coast Guard implemented internationally agreed upon safety standards for licensing and documentation of personnel on US seagoing vessels. ?? Recreational Vessel Regulations - 33 CFR 173, 174, 175, 179, 181, 183

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Coast Guard field units enter all compliance, enforcement and investigative actions into the Marine Information for Safety and Law Enforcement (MISLE) System, and program managers query MISLE to measure performance, set goals, and analyze the current "as is." For example, the Coast Guard analyzed towing industry deaths using MISLE data, which led to the joint American Waterways Operators and Coast Guard Safe Decks Campaign, "Stay Alert For the Edge"; Coast Guard district 17 developed a special effort for uninspected passenger vessels, and MSO Tampa carries out a special program for parasail operators. For the Recreational Boating Safety component of the Marine Safety program, annual boating accident data is collected from individual States and entered in the Boat Accident Report Database (BARD) System for tracking recreational vessel accidents and for performing analyses. In addition, States provide the Coast Guard with annual performance reports identifying individual program strengths and weaknesses. An annual PFD wear rate survey is conducted in 30 States.

Evidence: ?? USCG Port State Control Annual Reports (1998-2004): www.uscg.mil/hq/g-m/pscweb/Publication.htm ?? U.S. PSC Targeting Matrix: www.uscg.mil/hq/g-m/pscweb/Boarding%20Matrix.htm ?? QUALSHIP 21 Initiative: www.uscg.mil/hq/g-m/pscweb/Qualship21.htm ?? Annual Boating Statistics 1997-2003 www.uscgboating.org/statistics/accident_stats.htm ?? Annual PFD Wear Rate Survey - www.uscgboating.org/statistics/pfd.htm ?? Commonwealth of Virginia, Virginia Department of Game and Inland Fisheries, FY04 Boating Safety Performance Report

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Coast Guard's Officer Evaluation System (OES) includes a section that discusses individuals' results and effectiveness, and the evaluation form itself is customizable to identify individual contributions to achieving program goals. Based on the evidence provided, the Coast Guard could make accountability more explicit in the specific responsiblity descriptions. However, it appears that there is an appropriate level of accountability built into the managerial review process. Mechanisms to hold program partners accountable for program results are also not explicit, but do maintain an acceptable degree of accountability. The Coast Guard will revoke the certification of participants in the Alternate Compliance Program (ACP) and Streamlined Inspection Program (SIP) if compliance audits indicate failure to follow programmatic guidelines. Individual State programs are held accountable through annual financial audits for Federal grant funds and the proper administration of programs, but a direct link between grant awards and individual State program outcomes is still under development.

Evidence: ?? FY 2005 Performance Agreement between CG Commandant & DHS Secretary ?? Officer Evaluation Report (OER) Template, CG-5310 ?? Sample OER responsibility and achievement descriptions. ?? CG Marine Safety Manual, Volume II, Section B, Chapters 9 and 10 ?? 46 CFR Part 8, Alternate Compliance Programs ?? 33 CFR Part 96, Safety Management Systems ?? NVIC 2-95, CH-1; ACP ?? NVIC 2-99, SIP ?? Department of Land and Natural Resources, State of Hawaii, Financial Statements and Report of Independent Certified Public Accountants, June 30, 2004 ?? Commonwealth of Virginia's Quarterly Financial Status Report

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The Coast Guard obligates substantially all (over 99%) of its operating funds (Operating Expense Appropriation) each year. Virtually all capital acquisition funds (Acquisition, Construction, and Improvement Appropriation) are obligated prior to expiring. The Coast Guard's Office of Resource Management enforces the provisions of Commandant Instruction (COMDTINST) 7100.3 (series), Financial Resources Management Manual, that specifies quarterly spending rates and funding carry over limits. State grantees are provided with a categorical formula grant allocation each fiscal year for recreational boating safety. These grant funds are "no-year" dedicated money. Further, statutes allow for a State to have two years to obligate these funds. Should a State not obligate these funds within the two-year period, then the unobligated portion reverts to the next year's pot of grant funds to be distributed to all States. The Coast Guard tracks these obligations; and the State grantees provide quarterly Financial Status Reports of program expenditures and annual Performance Reports.

Evidence: ?? Estimated obligations by quarter in apportionments. ?? Department of Land and Natural Resources, State of Hawaii, Financial Statements and Report of Independent Certified Public Accountants, June 30, 2004 ?? Commonwealth of Virginia's Quarterly Financial Status Report ?? State Recreational Boating Safety Grant Allocations (FFY 2000-2005) ?? Southern States (SSBLAA) RBS Outlays (FFY 2000-2004)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The Coast Guard has developed an efficiency measure for the Marine Safety program that synthesizes the program's annual outcomes (number of marine casualties and deaths) with the cost to execute the program, adjusted for the effects of inflation. This measure has been incorporated into Departmental reporting mechanisms and is also being implemented throughout the agency. While the efficiency measure for the Marine Safety program appears robust, it will be important for the Coast Guard to use the appropriate inflationary adjustment to avoid overstating efficiency gains in the program. For example, the most recent computation of the efficiency of the Marine Safety program (2004) used a 3% inflationary adjustment even though the OMB-issued economic assumption for that budget year was 1.5%. In future computations, the Coast Guard should ensure inflation assumptions are consistent with budgetary assumptions. The Marine Safety program has not completed an A-76 study to date. However, the Coast Guard uses managerial best practices in areas such as information technology to promote efficiency gains within the program. The Marine Information for Safety and Law Enforcement (MISLE) system contributes directly to Marine Safety program cost effectiveness, and further gains are being realized with the use of Personal Data Assistants??to ease the administrative burden of recording inspection data and accessing voluminous laws, regulations, and policies.

Evidence: ?? MISLE brief to DHS JRC ?? CGDN+ brief to DHS JRC ?? DHS Investment Review Process, Management Directive (MD) 1400 (Excerpt) ?? Listing of CG Marine Safety & IT personnel support to DHS ?? Marine Safety efficiency measure explanation ?? Marine Safety Efficiency Measure and Efficiency Ratio Explained

Does the program collaborate and coordinate effectively with related programs?

Explanation: From the Commercial Vessel Safety perspective, the Coast Guard established formal partnerships with national and international maritime associations under its "Prevention Through People" strategy. These partnerships appoint industry and Coast Guard work groups to find solutions for human-factor problems. The Recreational Boating Safety component of the Marine Safety program is similarly engaged with program partners via extensive interactions with State boating safety programs, the Coast Guard Auxiliary, the U.S. Power Squadrons, the National Association of State Boating Law Administrators, the National Safe Boating Council (comprised of 351 Federal, non-Federal and private sector organizations), and the National Water Safety Congress (comprised of 116 Federal, non-Federal and private sector organizations).

Evidence: ?? Formal partnerships agreements and MOUs with various boating organizations and associations. ?? MOA Between the Commonwealth of Virginia and the USCG ?? MOU and Joint Action Plan for Employment of the USCG Auxiliary in Support of the Coast Guard's National Recreational Boating Safety Program

Does the program use strong financial management practices?

Explanation: The Coast Guard employs strong financial systems and techniques that meet or exceed the requirements of the Federal Accounting Standards Advisory Board. Although the recently-issued FY2004 DHS Inspector General audit report noted the need to improve Coast Guard financial reporting of real property and assets, there were no weaknesses specifically attributed to the Marine Safety program.

Evidence: ?? Financial Resource Management Manual, Commandant Instruction (COMDTINST) M7100.3C ?? USCG Finance Center Standard Operating Procedures (SOP) (TOC and Chapter 1) Standard Rates, COMDTINST 7310.1I ?? USCG Mission Cost Model ?? KPMG Audit of DHS Financial Statements for Fiscal Year 2004 ?? CFO Act Audit Remediation Strategic Plan, March 31, 2005 Including Appendices ?? Financial Reporting in the Office of Chief Financial Officer and DHS Components, Corrective Action Report, Issue and Milestone Schedule, April 15, 2005

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Coast Guard reviews its programs for management deficiencies both through a routine assessment process and more episodic evaluations. On an annual basis, Coast Guard executives develop Mission Planning Guidance, which highlights management initiatives to be implemented for the coming year. The Coast Guard also collects feedback on its regulations from the public to assess its already-issued regulations. In addition, the Coast Guard responds to internal feedback as well, such as when it conducted a performance analysis of its training for commercial vessel regulatory compliance examinations aboard U.S. and foreign flag vessels. The most far-reaching step to address management challenges is the agency's development of "Sector" commands, which realign all Coast Guard commands and assets in an operating area into a single command and control authority. This reorganization of the Coast Guard field structure will enhance accountability and performance of the Marine Safety program by integrating it with other prevention and compliance-related programs.

Evidence: ?? Commandant's Performance Excellence Criteria Message dated 17 October 2002 ?? Quality Performance Consultants, Commandant Instruction 5224.11A ?? Commandant's Quality Award Applicants & Results, 1996-2004 ?? MSO Jacksonville CQA Package (excerpt) ?? ALCOAST 290/05 Vertical Alignment of Headquarters, Area, and District Staffs ?? ALCOAST 105/04 Update on Coast Guard Sector Commands (SC) ?? ALCOAST 010/04 Establishing Coast Guard Sectors ?? U.S. Coast Guard Sector Implementation: An Overview, 3/05 ?? Marine Safety Reports Provided to Congress Since the Beginning of FY03 (on CD) ?? DOTIG Audit of the Performance Measure for the Recreational Boating Safety Program, April 20, 2000

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Coast Guard boat safety grant program is managed by the Office of Boating Safety, and designated Grant Managers oversee the activities of each grantee. The application process for the grant program is the first step to ensure accountability, and only states that meet the specified criteria may participate in the program. After funds are disbursed, quarterly financial status reports of program expenditures and annual performance reports are required from grantees. The Office of Boating Safety Regional Coordinators perform site visits to ensure State boating safety programs are in compliance with Federal requirements. In addition, as according to the Single Audit Act, when each State has their audit completed, if there are findings, the entity completing the audit forwards it, and the Coast Guard addresses the findings with the State to resolve any issues.

Evidence: ?? Department of Land and Natural Resources, State of Hawaii, Financial Statements and Report of Independent Certified Public Accountants, June 30, 2004 ?? Commonwealth of Virginia's Quarterly Financial Status Report ?? State Oversight Visit to Maryland Department of Natural Resources ?? State Recreational Boating Safety Programs, COMDTPUB P16755.3A (Grant Manual - CD Version)

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Quarterly Financial Status Reports of program expenditures and annual Performance Reports are required from grantees, and the Regional Coordinators in the Coast Guard's Office of Boating Safety perform site visits to ensure State boating safety programs are in compliance with Federal requirements. In addition, the Coast Guard collects data from each of the States on boating accidents, fatalities and injuries, and publishes an annual Boating Statistics report, which is accessible to the public via internet.

Evidence: ?? Department of Land and Natural Resources, State of Hawaii, Financial Statements and Report of Independent Certified Public Accountants, June 30, 2004 ?? Commonwealth of Virginia's Quarterly Financial Status Report ?? State Oversight Visit to Maryland Department of Natural Resources ?? Annual Boating Statistics 1997-2003 www.uscgboating.org/statistics/accident_stats.htm

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: The Coast Guard aggressively seeks public comment on its proposed rules, and in many cases, proposed rules are substantially changed based on views expressed by the regulated community. For example, Coast Guard dramatically cut the scope of the "Fire-Suppression Systems and Voyage Planning for Towing Vessels" due to industry comment: the cost of the rule dropped from $116 million to $19 million over the period of analysis (2003-2015). The Coast Guard also seeks regular input from the marine industry, the States, national boating organizations and the public through Coast Guard chaired forums such as the Merchant Marine Personnel Advisory Committee, the National Boating Safety Advisory Council, and the Towing Safety Advisory Committee. Membership in these groups includes industry representatives, the States, national boating, organizations, and the public; and the meetings are open to the public.

Evidence: ?? Final Rule, Fire-Suppression Systems and Voyage Planning for Towing Vessels, dms.dot.gov, docket number 6931 ?? Automatic Identification System: Vessel Carriage Requirement, AIS docket #14757 ?? Federal Register Regarding Automatic Identification System: Vessel Carriage Requirement, 1 July 2003 ?? Post Casualty Drug and Alcohol Testing, AIS docket #8773 ?? Federal Register Regarding Marine Casualties and Investigations; Chemical Testing Following Serious Marine Incidents, 28 February 2003 ?? Notice of Meeting, Towing Safety Advisory Council, Federal Register Feb 18, 2005 ?? Coast Guard Regulations: www.uscg.mil/hq/g-m/regs/reghome.html

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: Coast Guard's programs are well-supported by quantitative analysis that clearly identifies the impact of proposed rules. Every regulatory action contains cost benefit and willingness-to-pay analysis that meet OMB guidelines.

Evidence: All benefit/cost analyses associated with each rulemaking that Coast Guard conducts are located in the public docket at http://dms.dot.gov. Some examples follow: ?? Mandatory Ballast Water Management program for U.S. waters, docket # 14273; and ?? Maritime Transportation Security Act final rule cost/benefit analysis docket #14792. ?? Federal Register Notice of Proposed Rulemaking Regarding Mandatory Ballast Water Management Program for U.S. Waters, 30 July 2003 ?? Regulatory Evaluation, Mandatory Ballast Water Management Program for U.S. Waters, 15 July 2003 ?? Federal Register Temporary Interim Rule With Request for Comments and Notice of Meeting Regarding Implementation of National Maritime Security Initiatives, 1 July 2003

Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals?

Explanation: Coast Guard policies and procedures require program managers to conduct annual reviews to identify regulatory projects required to meet and ensure consistency in accomplishing program goals. The Coast Guard prioritizes these projects based on an evaluation of the programmatic impact of the proposal and input received from federal advisory committees, industry, major studies, comments from the public, casualty analysis, NTSB reports, changes to international codes and standards, and other reviews. In addition to this annual review process, the Coast Guard's regulations are continuously being reviewed and updated due to the interrelated nature of the marine safety regulations. When one section of the regulation is changed, other related sections will be reviewed to ensure consistency.

Evidence: ?? Coast Guard Headquarters Instruction M16703.1, Regulatory Development and Review, dated January 11, 1995, as modified by M16703.2, dated June 25, 2001. (Not in Binder) ?? Section 610 Reviews ?? Priority Ranking Tool ?? Regulatory Ranking Lists (November 2003, November 2004) ?? Review and Update of Standards for Marine Equipment, Federal Register Vol. 69, No. 125, June 30, 2004, p 39742. ?? 1995 Regulatory Reform project, streamlining initiative. Reviewed and revised several subchapters, instituting the Alternate Compliance Program (46 CFR Part 8), Lifesaving Appliances and Arrangements (46 CFR Subchapter W), and Marine Engineering (46 CFR Subchapter F).

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: Coast Guard regulations are developed and implemented due to identified safety problems not addressed by existing regulations and policies. The rulemaking process includes a cost benefit analysis, an impact analysis on small entities, and the burden of information collection analysis. The safety management system regulations are designed to be inherently flexible, and include specific alternatives for small entities to minimize the regulatory burden and maximize the net benefit. Coast Guard regulations are prevention-focused with enforcement provisions designed to reduce the need for response. In general, the analyses of the proposed rules show that prevention-related requirements offer far greater benefits relative to their costs than response-oriented requirements. The Coast Guard looks carefully at each component of a draft regulation, using incremental analysis, benefit/cost analysis, and the development of analysis of alternatives to maximize benefits across the entire rule. Only where specific solutions are dictated by statute are benefits not maximized.

Evidence: ?? 46 CFR Part 170, Small Passenger Vessel Inspection and Certification, Final Rule ?? 46 CFR Parts 20 and 15, Licensing and Manning for Officers of Towing Vessels, Notice of Proposed Rulemaking ?? 46 CFR Part 10, Radar Observer Endorsement for Operators of Uninspected Towing Vessels, Final Rule ?? 33 CFR Part 96, 46 CFR Pat 2, et al., International Management Code for the Safe Operation of Ships and for Pollution Prevention (International Safety Management (ISM) Code), Final Rule ?? 33 CFR Part 151, Implementation of the National Invasive Species Act of 1996, Notice of Proposed Rulemaking ?? 33 CFR Part 151, Penalties for Non-submission of Ballast Water Management Reports, Notice of Proposed Rulemaking ?? Recreational Vessel Regulations - 33 CFR 173, 174, 175, 179, 181, 183

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Boating deaths and casualties have fallen steadily over the past five years, however because the Coast Guard does not have a robust, long-term goal-setting process, it is impossible to judge whether the agency has made adequate progress relative to the program expenditures for that time period. While the Coast Guard is on track to meet its FY 2011 performance target, it is unclear whether that endpoint will be the result of programmatic achievement or statistical good fortune.

Evidence: ?? USCG Annual Performance Report & Budget In Brief FY 2005 ?? USCG Annual Performance Report & Budget In Brief FY 2004 ?? USCG Quarterly Performance Reports - DHS FYHSP ?? Graphs Showing Current Performance and Future Years Goals Related to Marine Safety ?? Charts and Graphs Showing Time-Based and Scenario-Based Average Annual Mainer and Passenger Deaths and Injuries ?? Graph of Recreational Boating Fatalities 1962 - 2003

Does the program (including program partners) achieve its annual performance goals?

Explanation: The Coast Guard collects a variety of data to support ad hoc analyses of short-term results, however there are no specific targets set for these "second-tier" annual measures, and the results are not published in any official way. As a result, it is impossible to tell whether the Coast Guard has achieved its annual performance goals, since goals do not exist. While the Coast Guard does set and consistently achieve the annual target it sets for the long-term measure it has established for the Marine Safety program, this metric is so broad that it makes determining the reasons behind annual progress difficult, if not impossible. The Coast Guard needs to do more to identify and promulgate throughout the organization a set of useful and informative performance measures that can be used by managers to understand short-term improvements or failures. Program managers claim that a limited number of annual performance measures are under development, and will function as a "dashboard" for program managers once implemented agency-wide. Once these measures are implemented on an agency-wide basis, it will be more apparent whether the Coast Guard is achieving annual performance goals.

Evidence: ?? USCG Annual Performance Report & Budget In Brief FY 2005 ?? USCG Annual Performance Report & Budget In Brief FY 2004 ?? USCG Quarterly Performance Reports - DHS FYHSP ?? Graph of Recreational Boating Fatalities 1962 - 2003

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Coast Guard has developed an efficiency measure for the Marine Safety program that synthesizes the program's annual outcomes (number of marine casualties and deaths) with the cost to execute the program, adjusted for the effects of inflation. This measure has been incorporated into Departmental reporting mechanisms and is also being implemented throughout the agency. While the efficiency measure for the Marine Safety program appears robust, it will be important for the Coast Guard to use the appropriate inflationary adjustment to avoid overstating efficiency gains in the program. For example, the most recent computation of the efficiency of the Marine Safety program (2004) used a 3% inflationary adjustment even though the OMB-issued economic assumption for that budget year was 1.5%. In future computations, the Coast Guard should ensure inflation assumptions are consistent with budgetary assumptions. The efficiency measure for the overall Marine Safety program in FY2003 was a favorable 1.10. The Coast Guard also reports an efficiency ratio for the Commercial Vessel Safety component of the Marine Safety program. The FY 2004 efficiency ratio for the commercial vessel safety (CVS) component showed a favorable 1.10; the ratio for FY2003 was a favorable result of 1.08.

Evidence: ?? USCG Mission Cost Model ?? USCG Marine Safety Performance Workbook ?? Marine Safety efficiency measure explanation

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Commercial Vessel component of the Marine Safety program compares favorably to other flag state compliance regimes. Coast Guard performance receives scrutiny to international standards under the Port State Control MOUs all over the world, i.e. Tokyo and Paris. These MOUs publish their evaluations annually. The U.S. fleet, inspected and administered by the Coast Guard, consistently equals or exceeds other flags for compliance evaluation, and meets or exceeds performance goals of the MOU organizations. The performance of the Recreational Boating Safety Program can be compared to an international counterpart with a similar social/economic/political environment - Canada. Given that the United States has a much larger boating population and a higher number of fatalities than Canada, a usable comparison is made by considering the number of recreational boating fatalities per 100,000 total recreational boats. In calendar year 2000, Canada had a recreational boating fatality ratio of 5.65 fatalities per 100,000 total boats, whereas the United States ratio was 4.36 fatalities per 100,000 total boats. In calendar year 2003, Canada had a ratio of 4.11 fatalities per 100,000 total boats, whereas the United States ratio was 4.08 fatalities per 100,000 total boats. The larger improvement made by Canada is thought to be significantly impacted by their implementation of a nationwide mandate on recreational vessel operator certification (mandatory education) beginning in 1999. Their requirement is a phased-in approach, whereby all motorboat operators will have to be certified by 2009. Through 2003, they report to have certified nearly 1 million operators. In the United States, while several States have operator certification requirements, we do not have a nationwide mandate.

Evidence: ?? Paris MOU Annual Report: www.parismou.org/ ?? Tokyo MOU Annual Report: www.tokyo-mou.org/ ?? USCG Model Maritime Service Code ?? Annual Report for Australian Coast Guard ?? 2003 USCG Port State Control Annual Rpt

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No comprehensive evaluation of the Marine Safety program has been conducted to date, so it is impossible to validate conclusions about program effectiveness. Although the Coast Guard plans to contract with the Center for Naval Analyses to review the Marine Safety program, that evaluation will not be completed until 2009 at the earliest. In the past, elements of the Coast Guard Marine Safety program has received third-party scrutiny by GAO and DHS IG auditors, industry partners, advisory groups, and organizations, international organizations and peer states. For example, Deloitte & Touche audited the Merchant Mariner Licensing and Documentation (MLD) Program in 2004, the Paris and Tokyo MOU organizations publish comparative performance statistics for member countries, and the DHS Inspector General audits Coast Guard mission performance on an annual basis. However, these analyses either do not encompass the whole of the Marine Safety program, as in the case of the Deloitte & Touche study, or the materials do not provide any managerial findings specific to the performance of the Coast Guard's program, as with the MOU and DHS Inspector General reports.

Evidence: ?? Evaluation of the USCG Quality Standards System (QSS) for the MLD Program by Deloitte & Touche, 2004 ?? Paris MOU Annual Report: www.parismou.org/ ?? Tokyo MOU Annual Report: www.tokyo-mou.org/ ?? DHS IG Audit OIG-04-43 FY2003 Mission Performance of USCG

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: There is no evidence that the Coast Guard conducts routine look-back analyses of its regulatory promulgations to validate the accuracy of its benefit-cost studies or the extent to which the regulatory frameworks are achieving results at least societal cost. There is anecdotal evidence that analysis that feeds the Coast Guard rulemaking process is robust. For example, GAO noted in a June 2004 report on Maritime Security (GAO-04-838) that while the Coast Guard had little time to develop its cost estimates, their own more detailed analysis found only a relatively small difference (<15 percent) in the estimated costs of the rule.

Evidence: ?? GAO report 04-838 (Appendix III): www.gao.gov/new.items/d04838.pdf.