Detailed Information on the
National Oceanic & Atmospheric Administration: Marine and Aviation Operations Assessment

OMAO will work to implement measures that guide and demonstrate improved efficiencies.

OMAO plans to perform more thorough analyses of alternatives for its capital acquisition activities to ensure that the program invests in assets that represent the best value to the government and the taxpayers.

OMAO will synthesize internal ship, aircraft, personnel and administrative policies to increase operational effectiveness.

OMAO will work to strengthen the link between performance results and program managers' personal performance evaluations to increase accountability and ensure annual and long term performance improvement.

Measure: Customer Satisfaction Rating for At-Sea Data Collection

Explanation:The Customer Satisfaction rating is calculated from scores assigned by projects' Chief Scientists and ships' Commanding Officers rating various aspects of a cruise experience. Ratings are assigned to areas including: safety, ship's physical plant, ease of efficiency of operations; which highlight areas of strengths and weaknesses. In addition to the numerical scores, a comments section in the survey provides the opportunity for respondants to suggest changes and/or improvement to OMAO leadership and ship support teams. Perfect = 4.0 Customer satisfaction data for at-sea data collection has been collected by OMAO Headquarters since at least 1992.

Measure: Customer Satisfaction Rating for Airborne Data Collection

Explanation:The Customer Satisfaction rating is calculated from scores assigned by projects' Chief Scientists and aircraft Commanders rating various aspects of a flight experience. Ratings are assigned to areas including: safety, aircraft's physical plant, ease of efficiency of operations; which highlight areas of strengths and weaknesses. In addition to the numerical scores, a comments section in the survey provides the opportunity for respondants to suggest changes and/or improvement to OMAO leadership and aircraft support teams. Perfect = 4.0 Customer satisfaction data for airborne data collection has been collected by OMAO Headquarters since 2000. The average score was 2.89 out of 3.0 for 2000-2002.

Measure: Ship Operating Day Accomplishment Rate

Explanation:Ship Operating Day Accomplishment Rate reflects the percentage of actual operating days delivered against the number allocated; perfect = 100% Ship operating day accomplishment rate has been collected by OMAO Headquarters since at least 1993. Historic accomplishment rates are as follows: 1993 - 95.6% 1994 - 97.3% 1995 - 98.5% 1996 - 97.6% 1997 - 95.0% 1998 - 90.6% 1999 - 93.4% 2000 - 93.2% 2001 - 91.5% 2002 - 93.9%

Measure: Aircraft Flight Hour Accomplishment Rate

Explanation:Aircraft Flight Hour Accomplishment Rate reflects the percentage of flight hours flown against the number allocated; perfect = 100%

Measure: Achieve major acquisition milestones of cost, schedule, and performance

Explanation:This measure is the percentage of the total number of major acquisition milestones identified in OMAO's Annual Operating Plan successfully met during the year. Monitoring the timely accomplishment of milestones identifies potential roadblocks and delays, prompting mitigation planning, if needed. Major acquistion milestones for cost, schedule, and performance are documented in the Quarterly Reports to the Deputy Under Secretary of Commerce for Oceans and Atmosphere; perfect = 100%. In 2007, OMAO will provide NOAA leadership with long-term recapitalization plans for NOAA ships and aircraft which will set long-term milestones for our capital planning process.

Measure: Wage Marine Attrition Rate

Explanation:NOAA Wage Marine Attrition Rate reflects turn-over of wage mariners who operate NOAA ships (crew). Retention of highly-qualified and trained wage mariners is essential to the safe, effective, and efficient operations of NOAA ships. Successful reduction of the attrition rate improves continuity and consistency of at-sea data collection and increases cost efficiency through reduced recruitment and training costs.

Measure: Lost-Time Accident Rate for vessel and aircraft data collection.

Explanation:This measure reflects the rate of accidents involving OMAO personel resulting in lost time (at least one full work day) and is defined and calculated in accordance with OSHA standards. An incident rate of 1 means that 1% of the workforce was affected by an accident resulting in lost time during the recorded period.

Measure: Recordable Accident Rate for vessel and aircraft data collection

Explanation:This measure reflects the rate of recorded accidents involving OMAO personel and is defined and calculated in accordance with OSHA standards. An incident rate of 1 means that 1% of the workforce was affected by a recordable accident during the recorded period.

Measure: Average cost per linear nautical mile of hydrography by NOAA ships

Explanation:Average cost of nautical charting. FY 2006 constant dollars will be used. The costs include all funds obligated to allow operation and maintenance of NOAA ships. This includes direct ship operations costs such as fuel, officer and crew salary and benefits and ship maintenance; shoreside support costs such as electronics and logistics support, OMAO Headquarters support costs; Commissioned Personnel center support costs. The baseline year for the cost per linear mile is FY 2006 using the FY 2005 actual rate adjusted to FY 2006 dollars. The target is to reduce the cost per linear nautical mile (in constant FY 2006 dollars) by 1% annually. Because FY 2006 is almost over, no reduction is included for the FY 2006 target.

Is the program purpose clear?

Explanation: The National Oceanic and Atmospheric Administration's (NOAA) Office of Marine and Aviation Operations' (OMAO) procures and operates ships and aircraft to support NOAA programs. OMAO provides data collection using vessels and aircraft and technical expertise to support NOAA's program outcomes including: the sustainable management of living marine resources, safe commercial and recreational navigation, coastal mapping, severe weather warnings and flood prediction, coastal and marine monitoring and preservation, and understanding of climactic, atmospheric, and ocean processes.

Evidence: OMAO mission statement contained in the 2007 Budget. OMAO's component activity charters (Marine Operations and Maintenance, Aircraft Services, Fleet Replacement, and Aircraft Replacement) define the requirements, roles and responsibilities, and beneficiaries of the program. NOAA's Strategic Plan defines the agency's goals and objectives.

Does the program address a specific and existing problem, interest, or need?

Explanation: OMAO provides technical expertise and data collection platforms that support NOAA programs to understand and predict changes in the Earth's environment and conserve and manage coastal and marine resources. OMAO contributes to the majority of NOAA's programs including, fishery conservation and management, nautical charting, weather forecasting and warning, research on global climate change, and ocean research and exploration. OMAO maintains unique data collection platforms. For example, the Ronald H. Brown is the only ship in the U.S. domestic fleet with a Doppler radar to conduct weather and climate research; NOAA's new ships Oscar Dyson and Henry B. Bigelow are the only U.S. Fisheries Survey Vessels designed to meet international standards for acoustic quieting for fisheries research; and NOAA's Gulfstream IV jet has highly specialized instrumentation for hurricane reconnaissance. OMAO also uses outsourced platforms and personnel to collect data. For example, UNOLS ships are used for ocean and climate research.

Evidence: 15USC Chapter 9 "National Weather Service" requires NOAA's National Weather Service to forecast the weather, issue storm warnings, report temperature and rainfall conditions, and take meteorological observations necessary to establish and record the climatic conditions of the United States. 16USC Chapter 31 "Marine Mammal Protection", 16USC Chapter 35 "Endangered Species", and 16USC Chapter 38 "Fishery Conservation and Management" make NOAA's National Marine Fisheries Service responsible for the management and conservation of living marine resources within the United States' Exclusive Economic Zone. The long-term collection of data at-sea in support of this effort requires the dedicated and technologically sophisticated ships designed, constructed and modernized by the Fleet Replacement Program. 33USC Chapter 17 Section 892 authorizes NOAA to "...procure, lease, evaluate, test, develop, and operate vessels, equipment, and technologies necessary to ensure safe navigation and maintain operational expertise in hydrographic data acquisition and hydrographic services??." 33USC Chapter 17 Section 891b authorized the acquisition of up to six (6) Fisheries Survey Vessels (FSV).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: OMAO provides data collection capability for NOAA's programs through a mix of specialized and outsourced platforms. NOAA's mission responsibilities, including nautical charting, fisheries management, climate research, and weather forecasting, necessitate data collection. OMAO acquires, operates, and maintains uniquely configured ships and aircraft to support NOAA programs, for example, acoustically-quiet Fisheries Survey Vessels (FSVs) that provide a unique capability not found elsewhere in the U.S. fleet (private or government) and NOAA hurricane hunter aircraft are uniquely equipped, for example with Doppler radars which supply critical data for hurricane intensity and track forecasting. OMAO works closely with federal partners to reduce overlapping efforts. There are other ship and aircraft acquisition and management programs that have similar characteristics; however, these programs support the different missions and mandates of their particular agencies. For example, the US Coast Guard supports safe navigation; however, they do not conduct nautical charting. The U.S. Navy performs hydrographic surveys; however, they are typically deployed in deep water, whereas NOAA concentrates on US coastal and EEZ regions. Similarly, the US Air Force collects tropical cyclone data which complements NOAA's hurricane reconnaissance activities. OMAO outsources for data-collection requirements beyond the capacity of its fleet where feasible. For example, OMAO utilizes excess capacity within the University-National Oceanographic Laboratory System (UNOLS) to complement NOAA's data collection requirements. The UNOLS fleet performs oceanographic research; however, their concentration is on basic research and they are not equipped to perform the requirements for long-term fisheries stock assessments and hydrographic surveys. NOAA's data needs also require that data collection be conducted in a range of environments, some remote and dangerous, and at all times of the day. The NOAA Commissioned Corps, composed of officers who have college degrees in engineering, earth sciences, oceanography, meteorology, fisheries science, and other related disciplines, command NOAA ships and aircraft and staff NOAA line offices. They provide NOAA with a cadre of highly-mobile, qualified individuals available on a 24 hour a day/7 day a week basis.

Evidence: 33USC Chapter 17 "National Ocean Survey" Section 883i authorizes NOAA to "??acquire, construct, maintain and operate ships??" and Section 892 authorizes NOAA to "??lease??and operate vessels, equipment, and technologies necessary to ensure safe navigation and maintain operational expertise in hydrographic data acquisition and hydrographic services??." 33USC Chapter 17 Section 891b directed OMAO to procure Fisheries Survey Vessels that meet the ICES standards for acoustic quieting. NOAA Ship RONALD H. BROWN is uniquely equipped in the national research fleet to perform state-of -the art atmospheric research. OKEANOS EXPLORER is specifically mentioned in the US Ocean Action Plan as the first ship solely dedicated to ocean exploration. The National Hurricane Operations Plan Chapter 2 and Chapter 5 identifies keys distinctions between Department of Defense and Department of Commerce (which includes NOAA) for tropical cyclone tracking and observation. NOAA hurricane hunter aircraft are uniquely instrumented for tropical cyclone tracking and intensity forecasting. The US Air Force is only required to provide hurricane reconnaissance; NOAA provides hurricane research and reconnaissance. The National Hurricane Operations Plan for 2006 will include the NOAA requirement for Stepped Frequency Microwave Radiometer and Tail Doppler Radar capability to support additional reconnaissance/research needs. 33USC Chapter 43 authorizes the establishment of a "commissioned officer corps" within the National Oceanic and Atmospheric Administration. The NOAA Commissioned Corps provides a cadre of professionals. Officers are trained for positions of leadership and command in the operation of ships and aircraft; in the conduct of field projects on land, at and under the sea, and in the air; in the management of NOAA observational and support facilities; as members or leaders of research efforts; and in the management of various organizational elements throughout NOAA. Officers rotate among these assignments, bringing a rare combination of operational, technical, and policy expertise to their jobs. 16USC Chapter 31 "Marine Mammal Protection", 16USC Chapter 35 "Endangered Species", and 16USC Chapter 38 "Fishery Conservation and Management" make NOAA's National Marine Fisheries Service responsible for the management and conservation of living marine resources within the United States' Exclusive Economic Zone. The long-term collection of data at-sea in support of this effort requires the dedicated and technologically sophisticated ships designed, constructed and modernized by the Fleet Replacement Program. 15USC Chapter 9 "National Weather Service" requires NOAA's National Weather Service to forecast the weather, issue storm warnings, report temperature and rainfall conditions, and take meteorological observations necessary to establish and record the climatic conditions of the United States.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: OMAO's current program design is free of major flaws that would limit its effectiveness. OMAO works very closely with NOAA programs to plan for future requirements and allocate the current fleet among programs on an annual basis. Having a commissioned corps also adds flexibility and technical expertise in NOAA's ability to collect data to meet mission requirements. OMAO conducts several analyses of ship and aircraft operations (e.g. cost-effectiveness studies and cost comparisons of gov't-owned vs. competitively sourced data collection). For example, recent analysis of the aircraft operations have resulted in underutilized aircraft being removed from service. OMAO should ensure that modernization planning systemmatically incorporates cost benefit analyses of the fleet, to help to better identify when new aircraft and vessels are necessary; when to outsource data collection; to efficiently outfit platforms with instruments that can do several types of data collection; and consider unmanned aircraft and submersible systems for data collection.

Evidence: Capital asset acquisition activities have clear lines of authority, responsibility, and accountability. Major acquisitions comply with NOAA Administrative Order 208-3, which defines major system acquisition plans, policies and procedures. Through the Planning, Programming, Budgeting, and Execution System (PPBES) process and the NOAA Fleet Allocation Council, OMAO is continually improving the process and methodology to ensure effective platform allocation to the most critical/highest priority data-collection projects. Exhibit 300 of the Fishery Survey Vessels acquisition. Department of Commerce Inspector General report "Program for Acquiring Fisheries Research Vessels Requires Stronger Management Controls" (June 2002) findings and recommendations were implemented to strengthen the acquisition effort. KPMG Study "Cost Analysis of Hydrographic Surveys in Alaska and the Gulf of Mexico" (October 2001) evaluates the cost per square nautical mile of conducting hydrographic surveys using NOAA ships, contractors and time charters.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: In general, OMAO effectively targets resources towards its primary goals of safe and efficient data collection, maintenance of existing platforms, and replacement of older platforms. NOAA has several collaborative processes to ensure that limited resources are allocated to NOAA's highest priority data collection projects. The Program Planning and Budget Execution System (PPBES) defines how OMAO contributes to NOAA's annual, long-term, and strategic goals. This process helps to guide programmatic activities through an annual review of priorities and budget allocations to ensure the program is on track to meet its strategic objectives. As part of the PPBES effort, the NOAA Fleet Allocation Council, which includes NOAA Line Office representation, annually directs the allocation of operating days/flight hours on NOAA platforms. The Fleet Allocation Council reviews and approves ship and aircraft allocation plans, provides guidance and mediation when scheduling conflicts arise, and makes strategic decisions directing the usage of multi-mission platforms. The Council also provides oversight for OMAO capital acquisition activity - for example the Fisheries Survey Vessel (FSV) acquisition project is overseen by the Fleet Allocation Council. For future capital asset planning, NOAA program managers and platform operators are directly involved in the requirements definition, design evaluation and construction / conversion /modernization of the platforms, platform systems and mission equipment. This ensures that acquisition resources are directed at assets that best meet the broad spectrum of NOAA's data collection requirements. OMAO should conduct a more systemmatic capital asset planning process, examining the entire scope of its operation, with more comprehensive information about the condition of its fleet, to better inform the long-term and annual resource allocation decisions.

Evidence: The ship allocation plans and aircraft allocation plans demonstrate where resources are being targeted to satisfy NOAA program data-collection requirements. An instrumentation subcommittee under development will provide an integrated assessment of platform instrumentation suites desired on OMAO-managed platforms. Other NOAA groups which oversee the planned use of limited resources include the NOAA Research Council, NOAA Ocean Council, Unmanned Aircraft Systems (UAS) and Autonomous Underwater Vehicles (AUV) Steering Committees and working groups.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: OMAO is a data collection program for NOAA programs. Data collected by OMAO contributes to the outcome goals of NOAA's programs. For example, OMAO operated vessels and aircraft contribute to NOAA's outcomes of healthy and productive coastal and marine ecosystems and the reduced loss of life, injury, and damage to the economy with the provision of accurate and timely weather information. These outcomes are addressed more directly in other NOAA programs, for example the National Marine Fisheries Service and the National Weather Service. OMAO's performance measures are output in nature and focus in the following elements of its program: Relevancy of the data collected through customer satisfaction information; Timeliness of data collected through operating Day / flight hour accomplishment rate and customer satisfaction information; Retaining a highly qualified workforce through the wage marine attrition rate (the program also measures retention of the NOAA Corp workforce, however wage marine attrition is currently high, and this measure is being used to address this higher priority issue.) Safety of its vessel and aircraft operations through lost accident rate and recordable accident rate; Cost efficiency through the average cost of nautical charting per linear nautical mile; and Acquisition milestone achievement within cost, schedule, and performance. These performance measures are both annual and long-term. They contain annual and long-term (5 years or longer) targets. These measures demonstrate progress on OMAO's 4 strategic goals: 1. Provide the best value in ship, aircraft and technical integration services (ship/aircraft accomplishment rate and cost efficiency measures). 2. Be NOAA's focal point for all vessel and aircraft operations and technical integration, including outsourcing support (customer satisfaction measures). 3. Recruit and retain a highly adaptable, technically competent and diverse workforce (wage marine attrition measure). 4. Expand public and private partnerships to best meet customer needs and NOAA business objectives (cost efficiency and customer satisfaction measures). These measures meet the majority of OMAO's goals, however the program should consider developing a long-term milestone measure that addresses its capital asset planning process. This process examines the optimization options for the fleet instrumentation, condition, and geographic location and examines how other sources for cost-effective data collection fit into this plan. Milestones could include conducting cost-effectiveness studies on emerging technologies (e.g. UAS) and more systematically assessing the material condition of its fleet.

Evidence: The FY 2007 Budget Submission for OMAO's Operation and Maintenance Funds lists the impact of program adjustments on aspects of OMAO's performance measures. NOAA's Strategic Plan for 2006-2011, defines the agency's goals and objectives and includes OMAO's performance objectives, including, "Increase the number of ship operating days and aircraft flight hours that meet NOAA data collection requirements with higher customer satisfaction." OMAO strategic plan for 2003-2008 OMAO contributes to numerous long-term performance measures reported by other NOAA programs that measure progress towards outcomes listed in NOAA's Strategic Plan, which include "??reduced loss of life, injury and damage to the economy...safe, secure, efficient and seamless movement of goods and people in the U.S. transportation system??healthy and productive coastal and marine ecosystems that benefit society??." Chapter 2 of NOAA's FY2007 Budget Summary lists the Government Performance and Results Accountability (GPRA) measures, many of which rely on data collected on NOAA platforms. OMAO adopts NOAA corporate performance measures for safety as stated in NOAA Administrative Order 209-1 "NOAA Safety Policy" to promote a safe work environment for all employees and NOAA contractors.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The long-term targets are specific and quantified and are ambitious. The measures have baselines. The long-term performance measure targets for increased accomplishment rates and customer satisfaction scores are ambitious based on the average age of NOAA ships and aircraft. Half of NOAA ships are over 30 years old. NOAA customer satisfaction scores are consistently higher for safety and shipboard / aircraft personnel support than the average overall score, which reflects the success of NOAA platforms in accomplishing NOAA data collection requirements. However, customer satisfaction scores also reflect the impact of age on NOAA platform equipment; reliability ratings are typically lower than the average overall score. UNOLS customer satisfaction scores reflect similar trends for aging ships and ship equipment. The targets for the acquisition milestones measure demonstrate improvement. The targets for the wage marine attrition rate demonstrate improvement from an actual of 35% in 2005, to a target of 24% by 2010. A 24% attrition rate for wage marines is similar to the industry average for this workforce. The targets for the safety measures (lost time accident rate and reportable accident rates) improve by 3% each year. The targets for the cost efficiency measure show a 5% improvement from 2005 to 2011.

Evidence: NOAA 2007 Congressional Justification contains OMAO performance measures. NOAA Strategic Plan for 2006-2011, OMAO Strategic Plan for 2003-2008 In 2005, half of NOAA's ships were in excess of 30 years old. UNOLS Post-Cruise Assessment Report for 2004 identifies challenges their organization faces which are similar in nature to those OMAO encounters. Data on wage marine workforce.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: OMAO is a data collection program and its annual performance measures are largely identical to its long-term measures; annual measures demonstrate progress toward reaching long-term goals As described in Question 2.1, data collected by OMAO contributes to the outcome goals of NOAA's programs. These outcomes are addressed more directly in other NOAA programs. OMAO's performance measures are output in nature and focus in the following elements of its program: Relevancy of the data collected through customer satisfaction information; Timeliness of data collected through operating Day / flight hour accomplishment rate and customer satisfaction information; Retaining a highly qualified workforce through the wage marine attrititon rate (the program also measures retention of the NOAA Corp workforce, however wage marine attrition is currently high, and this measure is being used to address this higher priority issue.) Safety of its vessel and aircraft operations through lost accident rate and recordable accident rate; Cost efficiency through the average cost of nautical charting per square nautical mile; and Acquisition milestone achievement within cost, schedule, and performance.

Evidence: The Department of Commerce Annual Performance Plan lists numerous performance measures which rely on airborne and at-sea data collection. The FY 2007 Budget Submission for OMAO's Operation and Maintenance Funds lists the impact of program adjustments on aspects of OMAO's performance measures. By meeting specific annual performance measures, OMAO contributes to NOAA Strategic Plan outcomes. One specific example includes the unique instrumentation on NOAA's hurricane hunter aircraft, the outcome of which is the improved accuracy of hurricane track and intensity forecast. Annual Program Baseline Assessments (PBA) document the program's current status, 100% requirements, and gaps with alternative strategies for meeting requirements. Annual Program Plans document proposed long term program goals and activities within anticipated fiscal resources. Both list long-term and annual performance measures. Monthly Performance Reports document OMAO's progress towards annual performance goals. They track the accomplishment of NOAA and outsourced ship operating days, NOAA and outsourced flight hours, and major acquisition project milestones.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The annual targets are specific, quantified, and ambitious. The measures have baselines. The annual performance measure targets all show continued improvement and are reasonable for the program to achieve.

Evidence: FY 2007 Budget Submission for OMAO's Operation and Maintenance Funds list the impact of program adjustments on aspects of OMAO's performance measures OMAO Annual Operating Plans document annual performance targets and milestones. Monthly Performance Reports document OMAO's monthly progress towards annual performance goals. They track the accomplishment of NOAA and outsourced ship operating days, NOAA and outsourced flight hours, and major acquisition project milestones.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: OMAO's partners include several contractors it uses for data collection activities and maintenance activities of OMAO platforms. These contractors include the University-National Oceanic Laboratory System (UNOLS) ships, shipyards, aircraft overhaul facilities, mission equipment providers, and calibration facilities. UNOLS ships are the largest partner OMAO uses for outsourced data collection. OMAO uses the following to ensure that projects using UNOLS ships are linked to OMAO's performance goals: Memoranda of Understanding with the National Science Foundation (for NOAA to acquire UNOLS vessel services); Individual Support Agreements which contain project-specific deliverables for performance, schedule, and cost; and through cruise assessments of UNOLS projects. NSF conducts regular reviews of cruise assessments to ensure UNOLS activities meet the performance requirements of the program's customers. Contractors for large acquisitions, such as the Fisheries Survey Vessels (FSV), are required to provide Earned Value Management reports that measure their performance in achieving OMAO goals (including cost, schedule, and performance progress). All other OMAO contracts contain performance deliverables that are linked to the annual and long-term goals of the program and contractors provide monthly status reports of their activities performed for the program.

Evidence: MOU with the National Science Foundation for the UNOLS program and Individual Support Agreements for specific projects. NOAA contracts identify aspects of performance measures the contractors are supporting and require their demonstration of how they support them. Shipbuilders (FSV / SWATH) develop vessel construction milestones and support project tasks to meet those milestones.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: OMAO is an output-oriented data collection and capital acquisition program. Evaluations that focus on the effectiveness of the capital acquisition process and the effectiveness of OMAO's data collection platforms in meeting customer needs are appropriate. Evaluations of OMAO's capital planning and acquisition process, an assessment of light aircraft operations, and cost comparison studies of nautical charting and aircraft replacement have been conducted. These evaluations were independent, of sufficient scope in covering both vessel and aircraft operations, were recently conducted, and were of sufficient quality to support program improvements. OMAO is currently conducting an evaluation of its long-term capital asset planning process through the Center for Naval Analysis. This study should determine the optimal balance of outsourced versus NOAA-owned and -operated ships to collect required data as well as alternative means to collect the data (such as unmanned aircraft). OMAO should also examine the mix of instruments to optimize platforms. In 2004, the GAO assessed the implementation of capital planning principles at the VA, NOAA, NPS, and Bureau of Prisons. Aircraft Operations: In 2005, Conklin & de Decker conducted an aircraft requirements analysis of NOAA's Citation II which is used for aerial photography survey activities, marine sanctuary monitoring, and mapping following natural and man-made disasters. The analysis compared the requirements, acquisition and operating cost for the current NOAA aircraft and several alternative aircraft. The 2000 Mitretek study "NOAA Light Aircraft Operations" assessed NOAA's light aircraft operations and examined: current and projected in-house costs, benefits of NOAA light aircraft, and a comparison of in-house versus charter costs and benefits. Mitretek also identified efficiency improvements to assist in long term planning. The study investigated a broad range of airborne data collection issues, including trends and needs, alignment of allocation with operational realities, contraction options, and organization realignment. Vessel Operations: The 2002 DOC IG report "Program for Acquiring Fisheries Research Vessels Requires Stronger Management Controls" examined whether construction of the Fisheries Research Vessels (FSVs) was meeting performance goals for cost and schedule and assessed the effectiveness of NOAA's management controls over the acquisition program. Acquisition of FSVs are the largest current acquisition activity in the OMAO program (over $200 million, over several fiscal years, for 4 FSVs). This evaluation was comprehensive in examining OMAO's capital acquisition process for FSVs - including the Federal Acquisition Regulation requirements, the reporting and tracking responsibilities for contractor performance, the award fee plan, and the OMAO management structure. The October 2001 KPMG study "Cost Analysis of Hydrographic Surveys in Alaska and the Gulf of Mexico" analyzed the cost associated with conducting hydrographic surveys in Alaska and in the Gulf of Mexico under three different data collection scenarios (survey costs for in-house operations, contractors, and time charters). The analysis included all support costs associated with conducting surveys under each scenario, including planning, source evaluation boards, contracting, tide analysis, supplies, and final data processing. The 1998 Mitretek study "Hydrographic Survey Data Collection: Analysis, Conclusions, and Recommendations" was an assessment of the operational use of three government ships supporting hydrographic surveying; compared the cost of the gov't operation to potential contractor costs; and examined opportunities for technology transfer, collaborations, and training programs to improve hydrography.

Evidence: Conklin & de Decker (May 2005) provided potential aircraft options (e.g. CASA, King Air 350, King Air 1900D) to replace the Citation. GAO Report "Agency Implementation of Capital Planning Principles Is Mixed" (January 2004) evaluated NOAA capital asset acquisition management and made agency-wide recommendations for improvement. KPMG (October 2001) examined the cost per square mile of using different surveying methods. The study found that contract surveys had higher costs on average than in-house surveys or time-chartering of vessels??." The study determined that NOAA-owned and operated hydrographic vessels were cost effective compared to commercial or charter services. Mitretek (February 2000) reported "??NOAA should maintain a light aircraft fleet??NOAA should establish objective procedures through which fleet assets remained aligned with mission needs??helicopters should undergo immediate underutilization review for potential disposal??look for opportunities to increase the use of twin otter aircraft??NOAA should examine the benefits of standardizing on fewer platform types??." The majority of the recommendations have already been accomplished. Mitretek (October 1998) provided 23 conclusions and 40 recommendations which included "??establish multibeam sonar as the base system for all surveys??encourage ONCO (OMAO) to consider using the anticipated PMC (MOC-P) relocation as opportunity to homeport the NOAA ship RAINER in Alaska??NOS should further investigate the feasibility, cost, and benefits of timely shore-based data processing using satellite transmission??."

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: NOAA uses the Planning, Programming, Budgeting, and Execution System (PPBES) to define how OMAO contributes to NOAA's annual, long-term, and strategic goals. This process helps to guide programmatic activities through an annual review of priorities and budget allocations to ensure the program is on track to meet its strategic objectives. The annual budget requests include the full cost of attaining performance goals and all programmatic increases and new capital projects include specific performance information. Budget and performance data are integrated and the impact of funding decisions on performance is shown. The NOAA Budget request identifies the OMAO performance measures, e.g. ship operating days and aircraft flight hours, that will be funded by the base budget and describes how base-funded ship and aircraft support contributes to NOAA performance goals. Budget and performance data are integrated and the impact of funding decisions on performance is displayed. The budget request identifies the full resource needs for base activities and any requested initiatives, identifies the NOAA and Commerce strategic goals supported by the these activities, and identifies the incremental performance improvements that will be provided by any initiatives. Examples of program initiatives and their impacts include: ?? FY07 $4.1 million request for additional operation funds for NOAA ships will allow OMAO to deliver an additional 370 operating days. This will allow OMAO to provide, for example, increased data to preserve coral reefs in Hawaii and perform fishery stock assessments in Alaska. ?? FY07 $0.7 million request for the operation and maintenance of new aircraft instrumentation on the G-IV. This instrumentation enables OMAO to provide critical data required for the newest hurricane forecast and modeling tools. ?? FY07 $0.8 million request for crew safety and rotation will help improve crew retention. This request is explicitly tied to OMAO's annual accomplishment rate targets by providing more trained crew members to meet safe manning requirements. This improves the ability of, for example, NOAA ship RAINIER to continue providing hydrographic data that increases safety of navigation in Alaskan waterways and ports. Capital acquisition requests in the budget also explain the relationship of the acquisition to overall program performance goals and identify impacts of changes on program performance. For example, in the 2006 Budget, OMAO request $32M for construction of FSV-4, and explained that the FSV-4 would have state-of-the-art acoustic technologies, combined with very quiet radiated-noise signature, to enhance the effectiveness and efficiency of at-sea resource surveys. Because of these capabilities, FSV-4 would be able to monitor up to nine times more volume of water for the same time and distance traveled by NOAA's current ships.

Evidence: The FY 2007 Budget Submission for OMAO's Operation and Maintenance Funds contains the full cost of the program and describes the impact of base activities and initiatives on OMAO's performance measures.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The use of NOAA's Planning, Programming, Budgeting, and Execution System (PPBES) process has supported the identification and correction of strategic planning deficiencies. This annual process requires OMAO to develop plans that guide resource allocation decisions up to seven years in advance and reflect clearly defined and prioritized data collection requirements. For example, a recent PPBES process called for more at-sea data collection than could be met by the NOAA fleet of ships. This led to increased outsourcing for days-at-sea. In addition, some more costly and labor intensive ships were removed from service for utilization and efficiency reasons. Changes to airborne data collection requirements identified several underutilized aircraft which are being replaced by a more capable, preferred platform. As a result of the PPBES process, OMAO has also recently redefined its mission and vision statement to refocus the organization on core competencies. OMAO is updating its strategic plan and is benchmarking its performance against organizations such as UNOLS, as well as foreign government research fleets (such as Norway and Japan).

Evidence: The Annual Guidance Memorandum identifies agency priorities for the upcoming PPBES planning cycle. NOAA program requirements are revalidated annually and gaps between current resources and program requirements are documented, validated and prioritized. The resulting program plan aligns long-term plans with NOAA strategic goals and fiscal guidance expectations. The collaboration nature of the PPBES process strengthens the link between OMAO's outputs (at-sea and airborne data collection) and NOAA program outcomes.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: OMAO has conducted analyses of alternatives on several program components, however it does not have a long-term capital asset plan that takes a comprehensive look at the entire mission and fleet of OMAO, the appropriate mix of gov't-owned and outsourced data collection platforms, ways to optimize the instrumentation of platforms, and the use of unmanned submersible and aircraft systems. The Center for Naval Analysis is currently performing a "Data Collection Alternatives" study to determine the optimal balance of outsourced versus NOAA-owned and -operated ships to collect required data as well as alternative means to collect the data. Once complete, this study should help inform OMAO's long-term acquisition strategy for the program. Analyses of alternatives include the following: 2005 Conklin & de Decker in 2005 replacement study for NOAA's Citation aircraft. OMB Exhibit 300s for the FSVs from 2001-2003. 2001 KPMG study "Cost Analysis of Hydrographic Surveys in Alaska and the Gulf of Mexico" analyzed the cost associated with conducting hydrographic surveys in Alaska and in the Gulf of Mexico under three scenarios - NOAA owned/operated hydrographic vessels, contractor operations, and charter services. Mitretek's 2000 study "NOAA Light Aircraft Operations" assessed light aircraft operations and examined current and projected in-house costs, benefits of NOAA light aircraft, and a comparison of in-house versus charter costs and benefits. OMAO conducted an analysis of alternatives for the Fisheries Survey Vessel (FSV) program.

Evidence: OMB Exhibit 300 for FSVs. Several analyses of alternatives listed above on vessels for hydrographic surveys, fishery research, and aircraft.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: OMAO has beseline performance data and ambitious targets for its performance measures. OMAO collects flight hour and operating day rates that are reported monthly. Customer satisfaction surveys are required upon completion of each project to evaluate feedback on the quality of services. Average customer satisfaction scores are reported quarterly to the agency. The completed surveys are disseminated to appropriate marine and electrical engineering representatives for the ship identified in the survey, and, depending on the content of the feedback, additional training or redirection of funding are scheduled for the winter in-port period. For example, if surveys indicate problems with the reliability of a vessel, additional maintenance may be scheduled for the winter period. OMAO also collects cost data on at-sea and flight hour work. OMAO routinely analyzes these data and adjusts priorities or resources accordingly. For example, some more costly and labor intensive ships were removed from service for utilization and efficiency reasons. Analysis of airborne data collection requirements identified several underutilized aircraft which are being replaced by a more capable, preferred platform. OMAO also has a cost efficiency measure of the cost per linear nautical mile of hydrography. OMAO holds regular meetings and workshops with its customers (e.g. annual Fisheries Vessel Committee meetings, Fisheries Science Advisory Board) to discuss program performance, define requirements, and identify potential operational efficiencies. The feedback received from NOAA programs are used to improve customer satisfaction. For example, the NOAA Heavy Aircraft Conference validated the continued need for OMAO to support operation of the WP-3D aircraft. Fishery survey vessel (FSV) construction progress is reported monthly by the shipbuilder to the OMAO acquisition manager and includes earned value management data. FSV cost information is provided via monthly invoices and incorporated into an earned value management tracking and reporting system. Quarterly resource reviews are conducted on all capital asset acquisition and operational programs. Close monitoring of construction schedules allows the government, for example, to anticipate and respond to changes in staffing requirements (i.e. delays in delivery allows a delay in hiring and staffing actions).

Evidence: OMAO Annual Operating Plans document annual performance targets and milestones for which performance metrics, such as accomplished operating days and flight hours, are collected. The OMAO Customer Satisfaction Surveys for ships and aircraft seek feedback from NOAA programs that use NOAA platforms. These surveys are requested from the project leader at the completion of the project. The OMAO-sponsored NOAA Heavy Aircraft Conference resulted in the publication of the NOAA Heavy Aircraft Report. This report documents the results of the meeting and the influence the conference had on OMAO long-term management of the NOAA Heavy Aircraft. NOAA Light Aircraft Workshop results document another example of OMAO working together with its partners to gather information to manage and improve program performance. The Fisheries Survey Vessel contract requires contractors to report on their accomplishments versus their plan. NOAA Administrative Order 201-101 defines the scope and purpose of the NOAA Fleet Allocation Council, which ensures that NOAA platforms are allocated to meet NOAA program requirements. Feedback is provided by the membership of the Fleet Allocation Council to OMAO, which is used to improve performance.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: OMAO has identified the managers who are responsible for achieving key program results. Operational performance is the responsibility of the director of OMAO, the director of the marine and aviation operations centers, the director of the commissioned personnel center, as well as the commanding officers of ships and aircraft. These managers have performance standards that are incorporated in their individual performance plans, however OMAO is currently developing a more structured accountability process that links the achievement of OMAO performance goals to a manager's performance plan. Manager's performance standards will be more clearly defined and specific and tied to planned ship/aircraft operating plans and linked to the OMAO's performance goals and measures. Manager's funding control responsibilities will also be more clearly incorporated, by requiring a standard as to how well managers met their annual financial plans. Until these performance standards are developed and deemed adequately specific and quantifiable, a no is warranted for this question. OMAO also plans to participate in the Department of Commerce's pay-banding demonstration project as a method of more directly holding managers accountable for performance. Program partners include charters of non-NOAA data-collection platforms (UNOLS ships) and contractors. UNOLS ship activities are held accountable through an MOU and specific project plans with NSF, that have cost, schedule and performance standards. Cruise assessments are conducted on these activities, which document accomplishment, performance, and cost of the contracted services. Contracts contain minimum performance thresholds, for example the commercial and government minimum acceptance criteria for ship and instrument performance, as well as delivery dates and cost requirements. OMAO uses fixed price contracts. Large acquisition contracts, e.g. the fishery survey research vessels, have an award fee component as an incentive for good performance. Other maintenance contracts contain penalty clauses if cost, schedule, and performance milestones are not met. Contractors are required to report on progress meeting cost, schedule, and performance deliverables. Contracting Officer's Technical Representatives are assigned to all large contracts, and a formal evaluation process is used to address any performance issues with contractors.

Evidence: NOAA Business Operations Manual. Examples of manager performance plans. Fisheries Survey Vessel contract, Section H requires contractors to report on their cost and schedule accomplishments. Examples of maintenance contracts. UNOLS MOU with the National Science Foundation, examples of UNOLS specific project plans, UNOLS cruise assessments.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: OMAO establishes monthly spending plans by object class at the beginning of each fiscal year. These obligation plans are aligned with OMAO's goals because they are developed using the NOAA PPBES process. The PPBES process defines how the OMAO program contributes to NOAA's annual, long-term, and strategic goals. This process helps to guide programmatic activities through an annual review of priorities and budget allocations to ensure the program is on track to meet its and NOAA's strategic objectives. OMAO also uses a fleet allocation process, in which it solicits project requests and priorities from NOAA's programs for a given fiscal year, and then allocates the use of the vessel and aircraft fleet accordingly. Spending plans also reflect this process, as well as NOAA appropriations. OMAO evaluates monthly and quarterly expenditure rates against targets comparing actual vs. planned obligations, explains variances, and develops action plans to address the cause(s) of variances. OMAO managers are required to closely monitor progress of their projects and execution of program funds and take corrective action to mitigate risk and realign funding as appropriate, and address funding issues immediately. Analyses of quarterly expenditures (actual obligations against the plan) are reported to NOAA and DOC leadership quarterly. Variances are typically caused by weather delays or unscheduled maintenance. OMAO does not have single audit findings. OMAO also uses an invoice processing system to track actual expenditures against obligations and tracks commitments (i.e. when the program has committed to a project, but before obligations have been made), to keep close track of program funds. OMAO reports any large (over $10K) acquisition activities to the NOAA Acquisition Officer in a timely basis. The NOAA Acquisition Officer reports into the Federal Procurement Data System, as necessary, in a timely basis. OMAO's obligations have been consistent with its spending plans. OMAO obligated nearly 99% of its plan for its salaries and expenses account over the past five years. OMAO obligated an average of 80% of its plan for procurement and acquisition activities over the past five years. This is due to acquisition programs, particularly the fishery survey research vessels, that require more than one year to complete. This rate of obligation is appropriate for these types of capital acquisition activities. Spending plans for program partners (e.g. charter of non-NOAA data-collection platforms, and contracts) are developed that are aligned with OMAO's and NOAA's goals as a result of the PPBES process. OMAO tracks obligations for these activities against the plan, ensures funds are obligated in a timely manner, and takes prompt action to address any funding issues that may arise.

Evidence: The Planning, Programming, Budgeting, and Execution System (PPBES) is the process that the NOAA uses to link NOAA's strategic vision with programmatic detail, budget development, operating activities and information technology. NOAA and OMAO Strategic Plans Annual Spending Plans (by month / quarter) Commerce Business System Tracking Accounting for Platform System (TAPS) for in-house accounting to ensure CD435s are obligated in a timely manner OMAO monthly resource reviews of planned vs. actual expenditures Quarterly obligation reports, reported to NOAA and DOC leadership. "Report on Budget Execution and Budget Resources" (SF-133) OMB Circular A-11

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: OMAO has an efficiency measure with baselines and targets of hydrographic survey activities (cost per linear nautical mile of hydrography). The long-term target is to achieve a 1% annual reduction in the cost per linear nautical mile through FY 2011. This measure helps the program introduce more efficiencies into its hydrographic surveying operations, including initiatives to reduce attrition rates of vessel crew, optimize instruments across all vessels, and streamline other support costs. OMAO's acquisition activities include contract incentives encouraging and rewarding optimal performance. Earned Value Management reporting is used in the FSV acquisition project, OMAO's largest capital asset acquisition project. OMAO aircraft programs utilize Activity Based Costing (ABC) and performance management in evaluating the cost per flight hour, cost per gallon of fuel per flight hour, and overall cost per project to measure and achieve efficiencies and cost effectiveness in program execution. OMAO intends to expand the use of ABC to ship operations. OMAO monitors ship day rate trends for similarly-classed vessels, such as UNOLS ships, to compare cost trends. This comparison is used by OMAO to assess its cost trends to ensure that its operations are cost-efficient in comparison to a benchmark organization. These analyses indicate that OMAO is competitive with this class of operations. UNOLS ships are also used by OMAO for several projects. UNOLS represents a cost-effective option for OMAO to conduct projects that cannot be accomplished by OMAO's fleet. OMAO also measures the cost-effectiveness of specific OMAO data collection services against alternative scenarios. For example, several hydrographic services cost comparisons of government platforms, time-charters, and contracting out, have been performed. OMAO has also conducted cost comparisons of alternative aircraft for its Citation II. OMAO is also undertaking cost effectiveness studies for new technology integrations (for example, for unmanned underwater and aircraft systems). These studies will examine the costs and benefits of incorporating these new data collection platforms into OMAO's operation. OMAO is working jointly with NASA on pilot projects and analysis of unmanned aircraft systems.

Evidence: NOAA Congressional Justification. In the past two years, through the use of ABC and performance based evaluation, three aircraft were determined to be inefficient and no longer cost effective for the limited missions supported by NOAA. These aircraft were sold and will be replaced with an aircraft that is more cost effective and efficient while filling a critical gap in fulfilling NOAA requirement for flight hours supporting mission goals. Cost data of UNOLS ships. Cost comparison studies of hydrographic survey operations and aircraft replacement.

Does the program collaborate and coordinate effectively with related programs?

Explanation: OMAO collaborates and coordinates with related and complimentary programs in areas such as fleet renewal, emerging technology development, and severe weather response preparation. This collaboration allows OMAO to take advantage of other Federal agencies best practices, prevents duplication of efforts, and coordinates roles and responsibilities in several areas. For example, on aircraft issues, OMAO: ?? collaborates with the National Aeronautics and Space Administration (NASA) on a joint pilot demonstration of Unmanned Aerial Systems (UAS) and is working on an MOU with NASA to continue this effort. This allows OMAO to learn a great deal about an emerging technology that could be used as a future data collection platform, while sharing the cost of research and development. ?? participates in the annual Interdepartmental Hurricane Conference (IHC) with the US Navy, NASA, US Air Force, and other Federal agencies, which develops the annual National Hurricane Operation Plan (NHOP). NHOP coordinates and formalizes interagency roles and responsibilities for hurricane research, surveillance and reconnaissance, to minimize duplication of effort and allocate resources effectively. For example, the NHOP describes the specific NOAA aircraft hurricane reconnaissance responsibilities and US Air Force hurricane reconnaissance responsibilities. ?? participates in the Interagency Committee on Aviation Policy (ICAP), which coordinates Federal agency aircraft policies. ICAP has implemented a number of programs to monitor and improve Federal aviation safety practices. On ship issues, OMAO: ?? consulted with Military Sealift Command (MSC) and the US Navy for design and pricing expertise for the development of new platforms such as the fishery survey vessel (FSV) and small waterplane area twin hull (SWATH). OMAO also has an MOU with MSC that leverages additional MSC expertise in contracting management for hydrographic services. ?? employs the US Coast Guard to inspect NOAA ships under construction to ensure they are being constructed to meet required safety and environmental standards. The US Coast Guard also inspects chartered ships to verify their compliance with Coast Guard regulations to ensure safety for NOAA personnel. ?? works with Office of Naval Research, National Science Foundation (NSF), Oceanographer of the Navy, and other Federal agencies on the Federal Oceanographic Facilities Committee. This group is currently developing an interagency plan to optimize the moderization and future configuration of the Federal oceanographic fleet. ?? incorporates aspects of vessel studies and designs acquired by the US Navy and NSF, in modernization plans of the OMAO fleet. OMAO also contributes to annual performance measures reported by other NOAA Programs. Among these are: ?? number of fishery stocks assessed, ?? number of square nautical miles surveyed, ?? increase warning lead time for severe weather and flooding dangers, ?? amount of marine debris removed from coral reefs, and ?? increasing public awareness of climactic, atmospheric, and ocean processes.

Evidence: Draft MOU with NASA on Unmanned Aircraft Systems. Annual National Hurricane Operation Plans. MOU with Military Sealift Command, MOU with US Coast Guard. OMAO participates on several interagency committees and councils such as FOFC, UNOLS, Interagency Coordinating Committee on Airborne Geosciences Research and Application, Interagency Committee on Aviation Policy, the Interagency Committee on Ocean Science and Resource Management. NOAA Congressional Justifications

Does the program use strong financial management practices?

Explanation: OMAO has a clean audit opinion and is free of material internal control weaknesses reported by auditors. Further, there are no issues with other non-compliance with laws and regulations related to financial management. OMAO follows prescribed Department of Commerce-wide financial management and accounting policies, procedures, and controls, and uses the Commerce Business Systems (CBS) as its financial management system. CBS is compliant with the Federal Financial Management Improvement Act and permits daily review of financial transactions. OMAO also uses other fund accounting tools (e.g. tracking accounting for platforms systems, construction work in progress) to maintain an accurate financial picture. As described in question 3.3, spending plans are aligned with program goals, OMAO managers closely track actual vs. planned spending to ensure obligations are made for intended purposes, ensure obligations are timely, and take prompt corrective action, as necessary. OMAO holds quarterly resource and capital asset reviews to monitor the status of projects and operating plans. Expenditure reports are reviewed quarterly by NOAA and DOC leadership.

Evidence: Commerce Business Systems (CBS) is a financial management system compliant with the Federal Financial Management Improvement Act and permits daily review of financial transactions. NOAA annual performance and accountability reports, which include independent audits of NOAA financial statements. Annual spending plans. Monthly and quarterly expenditure reviews.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: OMAO has a number of processes to identify and correct management deficiencies within its operational, acquisition, and HQ level activities. ?? Monthly, quarterly, and annual reports are internal evaluation tools that require managers to identify deficiencies, risks, cost variances, and corresponding corrective actions. ?? Quarterly Capital Asset Reviews monitor the achievement of acquisition milestones. They document deficiencies and require the identification of risk mitigation plans. ?? Customer satisfaction surveys results are distributed to OMAO managers to allow rapid identification and correction of operational issues. ?? Annual management conferences bring OMAO managers together to discuss safety, ethics, policy, and OMAO best practices. The goal of these conferences is to address specific management issues and identify solutions ?? Annual ship and aircraft inspections ensure that OMAO is properly managing and maintaining NOAA capital assets. ?? OMAO has used Operational Risk Management (ORM) for ships and aircraft. ORM is a management tool to determine the risks associated with new and routine operational projects. Management responsibilities and standard operating procedures that are monitored by these processes are established and documented within MOC DOCs and AOC Operations Manual. In addition to internal evaluations and procedures, OMAO engages in several external reviews of its management processes. ?? Monthly Fleet Allocation Council meetings provide third party oversight of OMAO program activity. The Council monitors OMAO's management with respect to the allocation of resources and planned acquisition activity. ?? Several Department of Commerce Inspector General (IG) and Government Accountability Office (GAO) reports have made recommendations affecting OMAO management procedures and processes, which were adopted by the program. For example, the recommendations provided by the IG report "Program for Acquiring Fisheries Research Vessels Requires Stronger Management Controls," discussed in Section 2.6, incorporated were: additional contracting resources to maintain official contract documentation in accordance with Federal Acquisition Regulations, monthly financial reporting procedures to improve cost controls, and award fee plans to encourage superior performance by contractors.

Evidence: NOAA Major project oversight - agency level management oversight of major acquisition projects. OMAO affected activity includes the FSV acquisition project. The OMAO Customer Satisfaction Surveys for ships and aircraft seek feedback from NOAA programs that use NOAA platforms. OMAO Customer Satisfaction Surveys, which are used to monitor the quality of service delivery, are available online on OMAO's home page. Management policies for the Marine Operations Centers managed by OMAO are documented within "MOC DOCs." The Aircraft Operations Center (AOC) maintains a similar collection of policies within its AOC Operations Manual. NOAA Administrative Order 201-101 defines the scope and purpose of the NOAA Fleet Allocation Council, which ensures that NOAA platforms are allocated to meet NOAA program requirements. It provides representatives from the various NOAA Line Offices the opportunity to provide feedback to OMAO from a management perspective. The establishment of the NOAA Aviation Safety program stemmed from an analysis and review of operations.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: OMAO major acquisition projects are managed in accordance with NAO 208-3, which requires acquisition strategies, plans, and approval prior to project development (actual platform design and appropriations requests). OMAO major acquisition projects, for example the Fisheries Survey Vessel project, have complied with NAO 208-3 by obtaining executive level approval at the following key decision points: ?? identification and definition of the mission need, its relative priority, and the funding requirements; ?? selection of competitive system design concepts to be advanced to a test/demonstration phase; and ?? commitment to full production. Individual acquisition contracts for capital assets, all fixed-price, include specific design and operational requirements. Contracts also include a schedule of deliverables with timeframes and selected government review responsibilities as well as performance tests and training of the NOAA crew prior to delivery. The use of fixed-price plus incentive contracts encourages cost effective procurements and timely delivery of products. Cost and schedule goals are clearly established and managed through Earned Value Management System reporting. Pre-defined milestones are monitored for timely attainment and form the basis for key decision points.

Evidence: NOAA Administrative Order 208-3 OMB Exhibit 300 for FSVs FSV contract

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Results are mixed for the program. In 2005, OMAO met its ship and aircraft accomplishment rate measures, its acquisition milestone measure, and its recordable accident rate measure. Trends in the ship and aircraft accomplishment measures, recordable accident rate measure, and cost efficiency measure show improvement. Specifically, improvement in ship and aircraft accomplishment rates has been in part due to new management procedures increasing accountability for achieving planned flight hours and operating days. However, historical trends are not available for several measures, and targets have not been met for the customer satisfaction measures and lost-time accident rate measure. Customer scores have remained constant. Customer satisfaction scores are consistently higher for safety and shipboard / aircraft personnel support than the average overall score, which reflects the success of NOAA platforms in accomplishing NOAA data collection requirements. Customer satisfaction scores also reflect the impact of age, which affects reliability, on NOAA platforms and equipment. Reliability ratings for ships and shipboard instrumentation typically lower the average overall customer satisfaction score. To improve the ability to meet the program's long term goals to increase accomplishment rates and increase customer satisfaction scores, OMAO has also done the following: ?? Has, since 1996, added 11 ships, modernizing and increasing capabilities of the NOAA fleet while allowing the retirement of 10 older, less efficient/capable ships from service. ?? Has added new, acoustically-quiet NOAA ships OSCAR DYSON and HENRY B BIGELOW (delivery June 2006), and is acquiring additional acoustically-quiet fisheries ships, which promote accurate fishery stock assessments to better manage the $43 billion US commercial and recreational fishing industry. ?? Has added NOAA ship RONALD H BROWN, the only U.S. oceanographic research ship equipped with a Doppler radar for advanced climate research to support climate change prediction models that forecast El Ni??o/La Ni??a events. ?? Has reactivated NOAA ship FAIRWEATHER to provide increased hydrographic survey capability, thereby increasing safety of navigation in US ports for the movement of $2 trillion in waterborne trade. ?? Is replacing the less capable Lake SeaWolf aircraft with a safer, more capable Twin Otter aircraft, which enhances capability to perform air chemistry research that evaluates air quality and large marine mammal surveys to protect and conserve endangered marine species. ?? Is working to incorporate autonomous underwater vehicles (AUVs); for example, an existing NOAA ship with added AUVs will be able to survey more nautical miles increasing safety in US waterways and ports. ?? Is adding to the Gulfstream IV aircraft a tail Doppler radar that will aid in improving hurricane track and intensity prediction models. ?? Is exploring, in partnership with NASA, the use of unmanned aircraft systems, enabling the collection of tropical cyclone data, for example, at altitudes at which it is too dangerous for NOAA personnel to fly.

Evidence: NOAA's Strategic Plan lists it's mission and goals as: (1) protect, restore, and manage the use of coastal and ocean resources through an ecosystem approach to management, (2) understand climate variability to enhance society's ability to plan and respond, (3) serve society's needs for weather and water information, (4) support the Nation's commerce with information for safe, efficient, and environmentally sound transportation, and (5) provide critical support for NOAA's missions. OMAO's Strategic Plan aligns directly with NOAA's Strategic Plan and lists long-term goals as: (1) data collection products and services delivered at the best value, (2) responsiveness to customer needs, providing reliable and safe operations, (3) improving quality of life for employees, and (4) strengthening partnerships with programs. DOC Annual Performance Reports contain OMAO performance targets and actuals. In April - November 2005, NOAA and NASA conducted the Altair Integrated System Flight Demonstration Project in cooperation with General Atomics Aeronautical Systems, Inc. (GA-ASI), to evaluate unmanned air vehicles (UAVs) for future science and operational requirements within the agencies, related to oceanic and atmospheric research, climate research, marine sanctuary mapping and enforcement, nautical charting, and fisheries assessment and enforcement. OMAO participates in a NOAA-wide analysis for integration of Autonomous Underwater Vehicles (AUVs) on NOAA ships. The analysis is examining the benefits and costs, and overall return on investment on AUVs to meet NOAA's at-sea data collection needs and long-term performance goals. The final report will be released in June 2006.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Similar to Question 4.1, performance results are mixed for the program. In 2005, OMAO met its ship and aircraft accomplishment rate measures, its acquisition milestone measure, and its recordable accident rate measure. Trends in the ship and aircraft accomplishment measures, recordable accident rate measure, and cost efficiency measure show improvement. Specifically, improvement in ship and aircraft accomplishment rates has been in part due to new management procedures increasing accountability for achieving planned flight hours and operating days. However, historical trends are not available for several measures, and targets have not been met for the customer satisfaction measures and lost-time accident rate measure. Customer scores have remained constant. Customer satisfaction scores are consistently higher for safety and shipboard / aircraft personnel support than the average overall score, which reflects the success of NOAA platforms in accomplishing NOAA data collection requirements. Customer satisfaction scores also reflect the impact of age, which affects reliability, on NOAA platforms and equipment. Reliability ratings for ships and shipboard instrumentation typically lower the average overall customer satisfaction score.

Evidence: OMAO's Annual Operating Plans document annual performance targets and milestones based on annual and long-term performance measures. DOC Annual Performance Reports contain OMAO performance targets and actuals.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: OMAO's cost efficiency measure, the cost per linear nautical mile of hydrography (held constant in FY 2006 dollars), was $9000 in 2005. This serves as the baseline target for 2006 and OMAO's goal is to reduce this cost by 1% annually, with a target cost of $8500 in FY 2011. OMAO has taken other steps to improve efficiency. Operational changes were made in the mid-1990s when a major study ("NOAA Fleet Management for the Twenty-First Century," October 1994) comparing OMAO and other ship operators demonstrated that OMAO was less cost-efficient. As a result of the study, several global class ships were retired from the fleet, and staffing requirements for the entire fleet were reevaluated and ship and shore complements were reduced. The result improved efficiencies in the cost per operating day. OMAO benchmarks day rate cost trends against similarly-classed UNOLS ships. Results show that OMAO is competitive with this class of operations. Likewise, cost comparisons with commercial hydrographic operators have shown OMAO to be competitive. OMAO also retired under-utilized aircraft when studies identified them as such, increasing the efficiency of aircraft operations.

Evidence: An Online Allocation Process is under development and will allow OMAO to enhance data acquisition activities and create a more efficient methodology to optimize allocation of operating days and flight hours. The internal study "NOAA Fleet Management for the Twenty-First Century" was completed in October 1994. This study made several major recommendations for program changes that have been implemented. A-76 on KA'IMIMOANA was successfully completed in favor of continued in-house operations in 1997. DOC Annual Performance Reports.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: OMAO acquires and manages uniquely configured data collection ships and aircraft to meet specialized data collection requirements for NOAA programs. OMAO compares itself to similar platform acquisition and operation programs such as UNOLS, MSC, US Air Force, and the US Coast Guard. For example, NOAA's ships provide more operating days per ship each year than other federal services. NOAA ship performance is comparable to the UNOLS research fleet which operates similar vessels for academic research. Targets for full operating year levels for each class/size of ship are similar. Cost comparison studies of hydrography surveying indicate that the cost of NOAA-operated hydrographic surveying is often less than contractor operations or time chartering. With respect to customer satisfaction scores, UNOLS uses a customer satisfaction survey tool like NOAA's to measure its responsiveness to customer needs and expectations. Positive trends in areas such as crew, safety, and logistical support compare between the two organizations; negative trends in ship and ship equipment reliability (both organizations have aging fleets) are also similar between OMAO and UNOLS. NOAA hurricane aircraft complete more taskings from the National Hurricane Center per aircraft than the US Air Force. In 2005, NOAA completed approximately 45 taskings per aircraft; the US Air Force completed approximately 25 taskings per aircraft. The results of all airborne data collection efforts are reflected in the Hurricane Season Tropical Cyclone Reports. OMAO compares its aviation activity to the industry standard in terms of cost per flight hour and to other Federal Agencies with aviation programs in terms of projects and mission completion. Favorable results in most areas were achieved. Where costs indicated a less efficient effort, aircraft were either sold or the project was completed by a charter aircraft.

Evidence: The UNOLS Fleet operates ships of similar class and endurance as the NOAA Fleet. The FY 2005 and FY 2006 UNOLS ship operating costs were approximately equal to the NOAA Fleet. The US Coast Guard's fleet of ships perform similar operating year levels and vessel class/size capabilities as compared to the NOAA Fleet. OMAO compares favorably with other federal agency scores as shown in the American Customer Satisfaction Index. The October 2001 KPMG study "Cost Analysis of Hydrographic Surveys in Alaska and the Gulf of Mexico" analyzed the cost associated with conducting hydrographic surveys in Alaska and in the Gulf of Mexico under three different data collection scenarios (survey costs for in-house operations, contractors, and time charters). The analysis included all support costs associated with conducting surveys under each scenario, including planning, source evaluation boards, contracting, tide analysis, supplies, and final data processing. The 1998 Mitretek study "Hydrographic Survey Data Collection: Analysis, Conclusions, and Recommendations" was an assessment of the operational use of three government ships supporting hydrographic surveying; compared the cost of the gov't operation to potential contractor costs; and examined opportunities for technology transfer, collaborations, and training programs to improve hydrography. "U.S. Navy Cuts Steaming Days, Aircraft Flight Hours" (Defense News 13Feb06) reported the US Navy's average for deployed ships is being reduced from 224 days per year to 156 days per year in 2006. NOAA's fleet of ships consistently average 215 days per year. US Navy aircraft hours are being reduced from 22.3 hours per month to 17.5 hours per month in 2006. NOAA's fleet of aircraft consistently average 22.9 hours per month.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: OMAO has had evaluations of the capital planning and acquisition process, an assessment of light aircraft operations, an assessment and cost comparisons of nautical charting, and an assessment of management controls in the acquisition of the fishery survey research vessels. In May 2005, Conklin & de Decker examined the costs and capability of alternative aircraft to NOAA's Citation II. OMAO is currently evaluating the alternatives recommended by the study to determine which option best meets NOAA's data collection requirements. In response to the 2002 IG audit recommendations, OMAO added additional contracting resources to maintain official contract documentation in accordance with FAR, initiated monthly financial reporting procedures to improve cost controls, developed and budgeted an award fee plan to encourage performance by contractors, and improved management control of the FSV acquisition program by defining project structure and responsibilities. The KPMG October 2001 study "Cost Analysis of Hydrographic Surveys in Alaska and the Gulf of Mexico" examined the cost per square mile of using different surveying methods as noted in Section 2.6. KPMG found that contract surveys had higher costs on average than in-house surveys or time-chartering of vessels..." The study determined that NOAA owned/operated hydrographic vessels were cost effective compared to commercial or charter services. The conclusions of this study and the Mitretek October 1998 study detailed below were influential in the decision to reactivate NOAA ship FAIRWEATHER and its assignment to the Alaska hydrographic survey mission. The February 2000 Mitretek study "NOAA Light Aircraft Operations" was an independent assessment of NOAA light Aircraft operations, as highlighted in Section 2.6. The study included 17 conclusions and 33 recommendations. Mitretek reported "??NOAA should maintain a light aircraft fleet??NOAA should establish objective procedures through which fleet assets remained aligned with mission needs??helicopters should undergo immediate underutilization review for potential disposal??look for opportunities to increase the use of twin otter aircraft??NOAA should examine the benefits of standardizing on fewer platform types..." The majority of the recommendations have already been accomplished, for example the disposal of the helicopters and the acquisition of additional Twin Otter aircraft to provide increased program support. Mitretek's October 1998 study "Hydrographic Survey Data Collection: Analysis,Conclusions, and Recommendations" validated the effectiveness of NOAA's hydrographic survey ships, triggered the exploration of the use of time charters, and led to the expansion of the use of contracts for hydrographic data. The study concluded that there was no economic basis for the immediate decommissioning of any of the three existing government ships currently supporting hydrographic surveying, unique Alaska work volumes, environmental conditions, and potential economic impacts presented a long-term need for deployment of hydrographic survey resources and private sector capacity and expertise to provide hydrographic survey services can only provide partial satisfaction of current NOS needs. OMAO has contracted with the Center for Naval Analysis to perform an analysis of the optimal balance of outsourced versus NOAA-owned and -operated ships to collect required data. The study "Data Collection Alternatives" is currently underway.

Evidence: Conklin & de Decker (May 2005) provided potential aircraft options (e.g. CASA, King Air 350, King Air 1900D) to replace the Citation. GAO Report "Agency Implementation of Capital Planning Principles Is Mixed" (January 2004) evaluated NOAA capital asset acquisition management and made agency-wide recommendations for improvement. KPMG (October 2001) examined the cost per square mile of using different surveying methods. The study found that contract surveys had higher costs on average than in-house surveys or time-chartering of vessels??." The study determined that NOAA-owned and operated hydrographic vessels were cost effective compared to commercial or charter services. Mitretek (February 2000) reported "??NOAA should maintain a light aircraft fleet??NOAA should establish objective procedures through which fleet assets remained aligned with mission needs??helicopters should undergo immediate underutilization review for potential disposal??look for opportunities to increase the use of twin otter aircraft??NOAA should examine the benefits of standardizing on fewer platform types??." The majority of the recommendations have already been accomplished. Mitretek (October 1998) provided 23 conclusions and 40 recommendations which included "??establish multibeam sonar as the base system for all surveys??encourage ONCO (OMAO) to consider using the anticipated PMC (MOC-P) relocation as opportunity to homeport the NOAA ship RAINER in Alaska??NOS should further investigate the feasibility, cost, and benefits of timely shore-based data processing using satellite transmission??."

Were program goals achieved within budgeted costs and established schedules?

Explanation: Performance targets have been largely met in OMAO acquisitions (e.g. FSV construction). There have not been significant cost overruns. The FSV deliveries have been delayed due to contractor bankruptcy and damage to the ship-building facility from hurricanes. OMAO met 9 of 11 major acquisition milestones defined in the FY05 annual operating plan. Two milestones targeted for FY05 - the completion of design and the start of construction on the planned SWATH ship - were delayed by 6 months because of damage caused by Hurricane Katrina to the design contractor's facilities.

Evidence: OMAO's Annual Operating Plan (AOPs) is a contract between the Director, Marine and Aviation Operations and NOAA Commissioned Corps and the Deputy Under Secretary for Oceans and Atmosphere. The plan stipulates ship and aircraft services to be accomplished (e.g., upgrading NOAA's Fleet of ship's and aircraft in accordance with Fleet Replacement Program funding) in support of continuous observation of critical environmental conditions. As mentioned above OMAO met approximately 80% of its FY 2005 milestones defined in the AOP, while 20% of its milestones were delayed due to Hurricane Katrina. The Quarterly Fleet Replacement Program report indicates progress in the design, construction or conversion of capital assets. The plan shows planned and actual obligations and their variances throughout the quarters. This report also explains met and unmet scheduled milestones. Key issues, risks and mitigation are also discussed. The report is reviewed quarterly by the NOAA Executive Panel and NOAA Executive Council. The Quarterly Capital Asset Resource Reviews are conducted to review the status of each major OMAO capital asset project and address upcoming issues. Participants include senior management and staff from the ship and aircraft centers, as well as headquarters. The FSV Acquisition Plan, approved in June 2000, established contract award and delivery dates as follows: Contract Award Delivery FSV1 Sep 2000 Oct 2003 FSV2 Feb 2002 Nov 2004 FSV3 Feb 2003 Nov 2005 FSV4 Feb 2004 Nov 2006 The total cost was estimated to be $227.2M for the four ship procurement. The actual dates achieved for the FSV are as follows: Contract Award Delivery FSV1 Jan 2001 Jan 2005 FSV2 Jul 2003 Jun 2006 (est) FSV3 Jan 2005 Oct 2007 (est) FSV4 Dec 2005 Sep 2008 (est) The total cost is estimated to be $232.6M for the four ship procurement.