Detailed Information on the
Hydrology Assessment

Develop and deliver the infrastructure and external communication procedures needed for community hydrologic modeling to increase efficiencies.

Provide new water resource forecast information in Internet-assessable digital form suitable for use by decision support assistance systems.

Measure: Economic benefits along river and lakes due to flood mitigation, enhanced water management, optimal irrigation scheduling and more efficient hydropower generation (Millions of dollars)

Explanation:The program provides emergency managers, water managers, other government agencies, and the general public with river and flood forecasts and warnings and water quantity information allowing them to improve their service delivery and resource allocation based on the best available information. The forecasts include uncertainty of occurrence information weeks in advance. Prior to FY 2000, these advanced services were not available.

Measure: Customer Satisfaction Index

Explanation:The American Customer Satisfaction Index (ACSI) is a uniform, cross-industry measure of satisfaction with goods and services available to U.S. consumers, including both the private and public sectors. The ACSI methodology is used within the Hydrology Program to identify areas of customer satisfaction and, more importantly, dissatisfaction. This will help the Hydrology Program address areas to improve services.

Measure: Implementation efficiency for establishing river and lake advanced hyrologic prediction capability (Thousands of dollars per location)

Explanation:The measure represents the cost (in thousands of dollars) to implement each river and lake forecast location with advanced hydrologic prediction capability. Current targets represent a 24% improvement in efficiency from FY 2003 to FY 2014.

Measure: River and lake sites with advanced hydrologic prediction capability (Forecast locations)

Explanation:The number of river and lake forecast locations where the probability of an event, such as river level, is associated with the forecast. When implementation is complete, advanced forecast information will be provided at 4,011 locations to assist emergency managers, water managers, and the general public to make decisions based on the probability of an event taking place.

Measure: Nationwide high resolution Forecast Coverage (Percent)

Explanation:Percent of the conterminous U.S. where high resolution (small geographic area) forecasts for surface and sub-surface water conditions are issued.

Measure: Flash Flood Warning - Lead Time (Minutes)

Explanation:Flash flood lead time is the difference between the time the warning was issued and the time the flash flood affected the area for which the warning was issued.

Measure: Flash Flood Warning - Accuracy (Percentage)

Explanation:Flash flood warning accuracy is the percentage of times a flash flood actually occurred in an area covered by a flash flood warning. The difference between the accuracy percentage figure and 100 percent represents the percentage of events that occurred without a warning.

Measure: Provide Hydromet Observation Data Delivery to the National Weather Service Field Offices (Minutes)

Explanation:Hydromet observation data delivery is the time required to acquire data from over 12,500 locations and process the data for distribution to the NWS operational facilities. Timeliness in delivering data contributes to the efficiency and accuracy for forecasts issued by the operational facilities.

Measure: Schedule Performance (Percentage)

Explanation:Schedule performance is the variance between the planned and actual dates for accomplishing milestones in the Hydrology Program Annual Operating Plan and 100 percent which represents all milestones being accomplished as planned.

Is the program purpose clear?

Explanation: The purpose of NOAA's Hydrology Program is to produce river level, flood and flash-flood forecasts and warning guidance, and water resource information that is disseminated to water and emergency response managers; other federal, state and local water management associations; and the general public throughout the country. The program includes the production and delivery of river flow forecasts and flood warning guidance, high-resolution analyses of precipitation, snow pack, and soil conditions, and the nationally consistent production of water supply estimates to fulfill the program's mission. Emergency managers use the forecasts and warnings for both strategic, long-term planning as well as tactical, short-term planning. Water resource managers make critical decisions that affect flood control, water supply, river and lake transportation, agriculture and hydropower.

Evidence: The NWS Organic Act (15 U.S.C. sec 313) (Evidence ID #47), provides the authority to forecast the weather and rivers, issue water related warnings, and collect and maintain related data. In addition, the Flood Control Act of 1938 (33 USC 706) (Evidence ID #57) authorizes the establishment, operation, and maintenance of a current information service on precipitation, flood forecasts, and flood warnings. Public Law 107-253, the Inland Flood Forecasting and Warning System Act of 2002 (Evidence ID #58) directs NOAA to conduct research and development, training, and outreach activities relating to inland flood forecasting improvement, and for other purposes. NWS policy directives define NWS activities and regulate its authorities and responsibilities. The Hydrology Program, as a subset of the Hydrologic Services Program, is governed by NWS policy directive 10-9 (Evidence ID #50). The Hydrology Program charter, developed for the NOAA's Planning, Programming, Budgeting, and Execution System, provides clear insight on the program's requirements, linkages to the NOAA strategic plan, outcomes, roles and responsibilities, and beneficiaries.

Does the program address a specific and existing problem, interest, or need?

Explanation: NOAA's Hydrology Program addresses the general need of society for advance information on river levels, floods, and water resources to save lives, protect property, enhance the economy, and conserve America's water resources. On average, flooding causes 87 deaths per year and an average of $4.7 billion in damage. The GAO Fresh Water Supply Report (2003) reports critical shortages in freshwater supply are imminent in 36 states within the next 10 years. Water managers rely on the forecast information provided by the Hydrology Program to optimize their operations, especially during shortages. As shortages grow, so will the demand for improved freshwater forecast information. It is estimated that when the Hydrology Program's Advanced Hydrologic Prediction Service is fully implemented, it will provide $766 million in economic benefits per year. The Hydrology Program meets the specific needs of the diverse group of users trying to address these problems in the following ways: ?? Local, State, and Federal Emergency Management Agencies and Organizations - Emergency managers at all levels of government need to make informed decisions for saving life and property. The Hydrology Program delivers timely and accurate river, flood and flash-flood forecasts enabling them to prepare and respond to these events. ?? General Public - The general public needs to make informed decisions for themselves and their families. The Hydrology Program delivers river, flood and flash-flood forecasts enabling the public to make informed decisions; whether using the water for recreation or for staying out of harm's way. ?? Water Management Agencies - These agencies need water quantity forecasts and warnings so they can optimize their water facility operations, e.g., reservoir management and hydropower scheduling. The Hydrology Program delivers short- and long-term water quantity forecasts to help water managers be efficient and effective. ?? River Commerce Transportation - This community needs river level forecast information for cost effective transport loading and scheduling. The Program provides river level forecast information to help shipping managers optimize their operations. ?? Agriculture and Industry - These groups need water availability information to make water allocation and use decisions. The Hydrology Program the program provides short- and long-term water supply information to address their needs.

Evidence: Flood fatality statistics (http://www.weather.gov/os/hazstats.shtml) for the 30 year period from 1975-2004 show an average of 87 people die each year in floods. Flood loss statistics (http://www.nws.noaa.gov/oh/hic/flood_stats/Flood_loss_time_series.shtml) show the annual average flood damage for a 20 year period (1983-2002) exceeded $4.7 Billion (adjusted to 2002 dollars). The May 2002 National Hydrologic Warning Council (NHWC) report, "Use and Benefits of the National Weather Service River and Flood Forecasts," indicates the Advanced Hydrologic Prediction Service (AHPS) will provide $766 million in economic benefits each year, once implemented throughout the United States.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: As authorized by the NWS Organic Act (15 U.S.C. SEC, 313), the Hydrology Program is the only national provider of flood watches; river and flash flood warning guidance; and river and lake level forecasts. The Hydrology Program is designed to work with NOAA and other Federal agencies, and state, and local governments to avoid duplication of effort and these relationships are outlined in various Memoranda of Understanding (MOU) and Memoranda of Agreement (MOA) and NWS directives. The Hydrology Program occupies a unique niche between the Federal, state, and local agencies who collect and provide data and NOAA's Local Forecasts and Warnings (LFW) Program which maintains the nationwide infrastructure to disseminate water, weather, and climate forecasts, warnings, and information to the public. These agencies, identified below, collect and provide the data which is then used by the Program in the production of daily river and flood forecasts and long-term water supply outlooks which are provided to the LFW program for public dissemination on a daily basis throughout the country. There are no known non-Federal (state, local, or private) efforts on a local or national scale to provide hydrologic forecasts on a routine basis to the LFW program for dissemination. 1. NOAA's Local Forecasts and Warnings program provides precipitation, temperature, and evaporation data and forecasts for use in the Hydrology Program's models. It also provides the official public product dissemination structure. 2. U.S. Geological Survey is the primary source for measurements of river height and flow data used in the program's hydrologic forecast models. 3. U.S. Department of Agriculture, Natural Resources Conservation Service, collects ground-based snow data used for the program's water supply forecasting. 4. U.S. Army Corps of Engineers and U.S. Bureau of Reclamation provide information on the quantity of water flowing through their reservoirs for use in the program's hydrologic models. 5. Numerous state and local agencies collect and provide data from local networks of precipitation, temperature, and river sensors. These data are used in the program's hydrologic forecast models. A 2003 National Research Council (NRC) report stated that the "current" model - in which the Federal government collects observations, develops and runs models, and generates forecasts and provides this information to the public, where the academic and private sectors can create specialized flood and river products, tools, and models - is appropriate and satisfies the government's obligation to make its information as widely available as possible. The program works closely with its partners to identify opportunities to share information and continues to systematically evaluate the need for new hydrologic and hydrometeorologic products and services to meet national needs, and develop a process for determining which products to discontinue.

Evidence: The program components work closely with other federal and state partners to leverage expertise and ensure efficiency, and have created an MOU between NOAA and US Army Corps of Engineers, an MOU between NOAA and US Geological Survey, an MOU between NOAA, the US Geological Survey and the Susquehanna River Basin Commission, and a Subsidiary Working Agreement between NOAA and US Department of Agriculture. NOAA and fellow agencies have come to agreement on the boundaries of responsibility for river modeling for specific agency missions in formalized Memorandums of Agreements (MOA) and Memorandums of Understandings (MOU). NWS Directive 10-9 defines the roles and responsibilities of the NWS entities in the development, production, and delivery of hydrologic services. The NRC report, Fair Weather: Effective Partnerships in Weather and Climate Services (2004) (http://www.nap.edu/catalog/10610.html), examines the roles of NOAA (primarily the National Weather Service [NWS]), academic institutions, and private companies in providing weather and climate services; barriers to interaction among the sectors; and the impact of scientific and technological advances on the weather enterprise. There are no programs in either the public or private sectors which duplicate the Hydrology Program's efforts to produce daily hydrologic forecasts and warning guidance to NOAA's LFW program and subsequent dissemination to the public throughout the country.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Hydrology Program is designed to produce, support, and enhance the hydrologic services delivered to the public by NOAA. The Program leverages data collected by other agencies with its own expertise to produce hydrologic forecast guidance products that are delivered to NOAA's Local Forecasts and Warnings (LFW) Program which maintains the nationwide infrastructure to disseminate water, weather, and climate forecasts, warnings, and information to the public. As designed, the program regularly meets its performance goals. The program design has three major elements: ?? The 13 watershed-based River Forecast Centers (RFCs), centers of hydrologic forecasting expertise in the Hydrology Program, produce daily river and flood forecast guidance based on continuous hydrologic modeling. They provide their guidance to the LFW program's Weather Forecast Offices (WFO) who disseminate the forecasts and warnings to the public. This is an effective partnership which provides the public with one outlet for official weather and water forecasts and warnings. ?? The NWS Office of Climate, Water, and Weather Services' Hydrologic Services Division (HSD) directs the hydrologic services program. HSD defines the requirements and policy, assesses performance, provides technical support for RFC and WFO hydrologic operations, leads the national customer outreach program, and serves as the interagency liaison for the overall hydrologic services program. ?? The NWS Office of Hydrologic Development (OHD) develops and delivers state-of-the-art hydrologic science and technology for RFCs and WFOs, Development and delivery follow the standard requirements-driven research-to-operations path, the Operations and Services Improvement Process (OSIP) established by NWS. OHD also manages the overall NOAA Hydrology Program which includes the Advanced Hydrologic Prediction Service (AHPS). The Hydrology Program design has evolved over the past 60+ years. The last major restructuring took place during the NWS Modernization and Associated Restructuring in the 1990s and was based on numerous studies of the most effective ways to provide hydrologic services. Several recent studies have examined various aspects of the Hydrology Program and made various recommendations for improvements, but none have recommended any major changes to the current program design. Although a complete analysis of the costs and benefits of the entire Hydrology Program has not been done, a 2002 the National Hydrologic Warning Council report entitled "Use and Benefits of the National Weather Service River and Flood Forecasts" estimated the successful implementation of the Program's AHPS and Water Resource components would result in $766 million in annual benefits (indexed to 2000 price levels) resulting in a benefit/cost ratio of about 12 to 1.

Evidence: The Hydrology Program and Local Forecasts and Warnings Program charters describe the program's requirements, outcomes, linkages to the NOAA strategic plan, roles and responsibilities, and users (http://www.ppi.noaa.gov/Weatherand_Water/ww_pdfs/Hydrology%20program.pdf and http://www.ppi.noaa.gov/Weatherand_Water/ww_pdfs/local%20forecasts%20and%20warnings.pdf). "Federal Hydrology (River and Flood Forecasting) Infrastructure" provides information and a flow chart defining the roles and interactions of federal agencies, enabling the NWS to provide river and flood forecasts. Execution reports and monthly quad charts contain performance results. OSIP is a formal, consistent, user-focused requirements-based management process to meet, current and future demands for National Weather Service (NWS) improved operations and services effectively and efficiently (http://www.nws.noaa.gov/directives/sym/pd01001curr.pdf). Hydrometeorological Service Operations for the 1990s (March, 1996) describes the major modernization of the NWS hydrometeorological services implemented during the 1990s. The NWS Modernization was a Congressionally approved and funded effort with associated oversight. "Use and Benefits of the National Weather Service River and Flood Forecasts", National Hydrologic Warning Council report (2002) looked at the costs and benefits of the full implementation of the AHPS program (12 to 1 ratio of benefits to costs). Additional evidence for the value of the entire range of water, weather, and climate services NOAA provides to U.S. households is cited in "Economic Statistics for NOAA, April 2006" which estimates that the average household pays about $13 per year for NOAA's weather services. Since the Hydrology Program budget is about 5% of the NWS budget, the average household pays less than $1 for hydrologic services. The same report cites total annual Federal spending for weather information (including defense, aviation, and NOAA) produces an annual benefit-cost ratio of 4.4 to 1 to U.S. households alone - without including benefits in agriculture, transportation, or construction. Studies of various aspects of the Hydrology Program which did not recommend major changes to the current program design include: ?? "Toward a New Advanced Hydrologic Prediction Service (AHPS)", National Research Council Report (2006). ?? "Toward Improving the NWS Hydrologic Services Program Today and in the Future", River Forecast Center Operations Team (April, 2003). ?? "The Northeast River Forecast Center Is Well Managed, But Some Improvements Are Needed", U.S. Department of Commerce's Office of Inspector General, Inspection Report No. IPE-17259 (2005). ?? The NRC report, Fair Weather: Effective Partnerships in Weather and Climate Services (2004) (http://www.nap.edu/catalog/10610.html) has identified the NWS as an efficient provider of weather and water services.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program's fiscal and human resources are used directly and effectively to achieve the program's purpose to deliver river level forecasts, flood and flash-flood warning guidance, and water resource information to be disseminated to the intended beneficiaries, the general public. This dissemination, through NOAA's Local Forecasts and Warning (LFW) Program, distributes the river, flood, and water information products in text and graphical format via methods such as the Internet, subscription-based Family of Services, and NOAA Weather Radio (NWR), to decision-makers, broadcast media, and end users for the public benefit. Leveraging the established dissemination infrastructure of LFW allows the Hydrology Program to target its resources on improving its hydrologic services - not establishing additional dissemination paths. The Hydrology Program has several components to its effort to ensure the hydrologic information reaches the widest possible number of intended beneficiaries. The Hydrology Program leads the national and regional customer outreach program establishing a dialog with groups such as the National Hydrologic Warning Council, the National Emergency Managers Association, and the Association of State Floodplain Managers to inform them of available hydrologic products and gather feedback. The Program also prepares and coordinates a set of outreach materials to be used by the LFW Program's Weather Forecast Office in their efforts to keep customers informed of the available hydrologic services. This helps ensure customers throughout the country receive quality, consistent information. There is also an identified need for the RFCs to become more engaged in customer outreach (IG Inspection Report No. IPE-17259). The Hydrology Program Manager is currently working with the NWS Corporate Board to enhance the communication and outreach function at every RFC. To determine the effectiveness of its hydrologic products and services, the Hydrology Program conducts a biennial customer satisfaction survey to identify successes along with areas for improvement. The survey has shown customer satisfaction with hydrologic services in nine main user groups to be higher than average for other Federal programs (78% vs. 71%). The program also continually solicits and evaluates customer feedback from its web pages to prioritize the need for improved or new water forecast information products and services. Although there are ongoing efforts to provide the hydrologic information in formats to make it easier for the private sector to add value to existing products and provide additional products, there has been no indication of a viable business case for them to step into a role of providing consistent, nationwide hydrologic services.

Evidence: Hydrologic Service Partners List contains the names of the national and regional partners the Hydrology Program works with, keeping them informed of the status and plans and gathering feedback on areas for improvement and collaborations. This is a dynamic list - names are added as new partners are identified and new connections are made. "The Northeast River Forecast Center Is Well Managed, But Some Improvements Are Needed", U.S. Department of Commerce's Office of Inspector General, Inspection Report No. IPE-17259 (2005) Recommendation 2 notes the need for increased RFC outreach. National Weather Service Customer Satisfaction Survey - Hydrologic Services Program (reports from 2004 and 2006) performed by Claes-Fornell International Group (http://www.cfigroup.com/). This survey used a version of the same methodology as the American Customer Satisfaction Index (ACSI). Fair Weather: Effective Partnerships in Weather and Climate Services (2004) (http://www.nap.edu/catalog/10610.html), examines the roles of NOAA (primarily the National Weather Service [NWS]), academic institutions, and private companies in providing weather and climate services; barriers to interaction among the sectors; and the impact of scientific and technological advances on the weather enterprise. It specifically recommends NWS data and products should be made available in Internet-accessible digital form in standards-based formats.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Long-term outcome goal: The Hydrology Program goal is to provide more timely and useful water information and forecasts to support improved decisions made by emergency managers, water resource managers, and the general public. Long-term outcome performance measures - These measures indicate the utility of hydrologic products and services. Providing valuable information is one aspect of addressing the program's long-term goal. Customer feedback is an essential tool for measuring the true impact of the programs' services. Surveys assist program managers in gaining information from users, which helps the program understand which services are valuable: 1. Economic benefits realized by communities residing near rivers and lakes due to improved flood mitigation, enhanced water management, increased agriculture, and more efficient hydropower generation. 2. Improving customer satisfaction with effective use of hydrologic forecast information. Long-term efficiency measure - This measure represents the cost to implement each river and lake forecast location with advanced hydrologic prediction capability: 3. Implementation efficiency for establishing river and lake advanced hydrologic prediction capability. Long-term output performance measures - These measures demonstrate the growth in the delivery of requested information (e.g., longer lead times, uncertainty information, information at finer resolution scales.) 4. Number of river and lake sites with advanced hydrologic prediction capability. 5. Percentage of nationwide high resolution forecast coverage.

Evidence: Long-term program goals and outcomes are documented in the Hydrology Program Charter and NOAA's Strategic Plan. Performance measures are documented in annual budget submissions and accompanying annual performance plans, line office strategic plans, and line office and program annual operating plans. Economic Benefits The program provides emergency managers, water managers, other government agencies, and the general public with river and flood forecasts and warnings and water quantity information allowing them to improve their service delivery and resource allocation based on the best available information. The forecasts include uncertainty of occurrence information weeks in advance. Prior to FY 2000, these advanced services were not available. American Customer Satisfaction Index (ACSI) The Hydrology Program assesses customer satisfaction through biennial surveys of ACSI. The ACSI is a uniform, cross-industry measure of satisfaction with goods and services available to U.S. consumers, including both the private and public sectors. The ACSI survey and analysis is performed externally and independently by a third party research and consulting firm specializing in the application of the ACSI methodology to individual organizations. The ACSI methodology is used within the Hydrology Program to identify the causes of customer satisfaction and relate satisfaction to performance measures. The Hydrology Program can then identify those factors that will most improve customer satisfaction and other measures of performance to maximize return on our strategic investment. References: NWS Customer Satisfaction Survey Hydrology Program Final Report 2004, NWS Customer Satisfaction Survey Hydrology Program Final Report 2006. Implementation Efficiency for Establishing River and Lake Advanced Hydrologic Prediction Capability The cost per river and lake forest location implemented with advanced hydrologic prediction capability. Current targets represent a 24% improvement in efficiency from FY 2003 to FY 2014. River and Lake Sites with Advanced Hydrologic Prediction Capability The number of river and lake forecast locations where the probability of an event, such as river level, is associated with the forecast. When implementation is complete, advanced forecast information will be provided at 4,011 locations to assist emergency managers, water managers, and the general public to make decisions based on the probability of an event taking place. Nationwide High-Resolution Forecast Coverage Percent of the conterminous U.S. where high resolution (small geographic area) forecasts for surface and sub-surface water conditions are issued.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Program aims to increase annual economic benefits along river and lakes due to flood mitigation, enhanced water management, optimal irrigation scheduling, and more efficient hydropower generation up to $766 million by FY14. The Hydrology Program is striving to improve the American Customer Satisfaction Index from 77 in FY04 to 80 in FY10. Claes Fornell International's (CFI) final report of the Hydrology Program Customer Satisfaction Survey from 2004 indicates that the ACSI score of 77 for the Hydrology Program is well above the federal government average of 71 and the industry average of 74 for customer satisfaction. According to the CFI customer satisfaction model for the Hydrology Program, the significant drivers of satisfaction are flood information, precipitation information, and water supply and reservoir information. For the Hydrology Program to improve its ACSI by one point, these drivers of satisfaction must be improved by almost five points. The program is striving to achieve a 24% improvement in efficiency in the cost of establishing river and lake forecast locations with advanced hyrologic prediction capability from FY 2003 to FY 2014. Based on efficiencies realized to date, the targets will be re-evaluated in FY 2007 order to improve their ambitiousness. The Program will increase the number of river and lake sites with advanced hydrologic prediction capability from 27 locations in FY00 to 4,011 locations by FY14. Since implementation began in FY00, estimated costs for the development have decreased from approximately $100 million to $70 million due to improved program efficiencies without changes in the deployment schedule. Through FY06, the program deployed 43% of the locations with advanced hydrologic prediction capability while expending only 38% of its allocated resources. The Hydrology Program will develop nationwide high resolution forecast coverage of surface and sub-surface water conditions including soil moisture. This new service will be provided for 35% of the conterminous United States by FY13. As each high resolution forecast coverage area is deployed, improved forecasting models will need to be calibrated with operational data for that specific area. As more models are operationally implemented, the new models will be a need to be continuously improved and integrated in order to capture the diverse hydrologic conditions of the United States.

Evidence: Performance measure targets are specified in annual budget submissions and accompanying annual performance plans, annual Performance and Accountability Reports, line office strategic plans, line office and program annual operating plans, and monthly/quarterly performance reports.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The annual goals of the Hydrology Program focus on efficient production of timely and accurate forecast information that provide useful water information and forecasts to support improved decisions made by emergency managers, water resources managers, and the general public. The Flash Flood Lead Time and Accuracy measures determine the amount of time and precision in the forecast information provided. The Delivery Time of Hydromet Data to Local Weather Forecast Offices measure focuses on the efficiency with which data is provided to field offices responsible for issuing official forecasts, watches, and warnings. The model used for the American Customer Satisfaction Index (ACSI) indicates that the leading driver for customer satisfaction and program effectiveness is the flash flood, flood, and routine river forecasts and information. By targeting and measuring improvements in these areas the Hydrology Program can demonstrate progress in achieving the program's long-term goals and improving customer satisfaction. The long-term output measures of the number of river and lake sites with advanced hydrologic prediction capability and the percentage of nationwide high resolution forecast coverage have annual targets that demonstrate the growth in the delivery of requested information. By meeting these targets the Hydrology Program shows progress in achieving the long-term goal to improve economic benefits realized by communities residing near rivers and lakes due to improved flood mitigation, enhanced water management, increased agriculture, and more efficient hydropower generation. The Schedule Performance represents the Hydrology Program milestones being accomplished as planned.

Evidence: The NWS Strategic Plan 2003-2008, the Annual Operating Plan and the NWS Services Improvement Plan identify annual targets for the Hydrology Program performance measures. Flash Floods (Lead Time & Accuracy) A flash flood is a rapid and extreme flow of high water beginning within six hours of the causative event (e.g., intense rainfall). The Lead Time metric is defined as the difference between the time the warning was issued and the time the flash flood event occurred. Flash Flood Accuracy is the percentage of time a flash flood actually occurred in an area covered by a flash flood warning. These measures allow other governmental agencies, emergency managers, first responders, and the general public to make informed decisions regarding flood control measures, evacuations, and resource positioning. American Customer Satisfaction Index (ACSI) The Program assesses customer satisfaction through biennial surveys of ACSI, a uniform, cross-industry measure of satisfaction with goods and services available to U.S. consumers. The ACSI survey and analysis is performed independently by a third party research firm specializing in the application of the ACSI methodology to individual organizations. The ACSI results are used within the Hydrology Program to identify the causes of customer satisfaction and relate satisfaction to performance measures. The Hydrology Program identifies those factors that will most improve customer satisfaction to maximize return on our strategic investment. References: NWS Customer Satisfaction Survey Hydrology Program Final Report 2004, NWS Customer Satisfaction Survey Hydrology Program Final Report 2006. Hydrometeorology Observation Data Delivery This is the time required to acquire and process data from over 12,500 locations and process the data for distribution to the NWS operational facilities. Each location has multiple sensors that transmit data several times per day. Timeliness in delivering data contributes to the efficiency and accuracy for forecasts issued by the operational offices. Schedule Performance is the variance between the planned and actual dates for accomplishing milestones in the Hydrology Program Annual Operating Plan. It represents the percentage of milestones being accomplished as planned.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Current goals are modest, but not ambitious. For most annual measures, actuals are outperforming targets, indicating that targets could be more challenging. The goal for flash flood warning lead time is to increase from 48 minutes in FY05 to 49 minutes in FY10 and for flash flood warning accuracy to increase from 89% in FY05 to 90% in FY10. Although these improvements are modest, significant improvements in lead time and accuracy are expected after the FY11 installation of Dual Polarization Radar. Targets for lead time and accuracy will be adjusted as this new technology becomes operational. There is a natural inter-annual variability for both lead time and accuracy. Analysis of verification data compiled since 1986 reveals that the lead time has been within one standard deviation (~4.3 minutes) of the mean target (adjusted to account for the implementation of NEXRAD) in all but 5 years. Additionally, with improved efficiency HADS data acquisition and processing time is targeted to decrease to 3.0 minutes by FY09 even with continually increasing data loads. The Program is also planning to improve schedule performance from 95 % in 2005 to 96% in 2010.

Evidence: Performance measure targets are specified in annual budget submissions and accompanying annual performance plans, annual Performance and Accountability Reports, line office strategic plans, line office and program annual operating plans, and monthly/quarterly performance reports, GRPA Target Revisions FY 2006, NWS Customer Satisfaction Survey Hydrology Program Final Report 2004, NWS Customer Satisfaction Survey Hydrology Program Final Report 2006. Flash Floods (Lead Time & Accuracy) The Flash Flood Lead Time increased by approximately 20 minutes and Accuracy increased from 46% to 84% between 1994 and 1997. This dramatic increase was primarily the result of installing Next Generation Radars (NEXRAD) which provided improved rainfall information. Significant improvements will be realized again after the installation of Dual Polarization radar in FY 2011. This enhancement will provide estimates of precipitation type consequently improving precipitation estimation. In the mean time, incremental improvements are being made through the implementation of the Flash Flood Monitoring and Prediction (FFMP) and forecaster training. FFMP is a decision assistance tool provided to Weather Forecast Offices that continuously monitors rainfall rate and accumulation, compares these with flash flood guidance, and provides the forecaster a visual indication and automated alerts when a flash flood situation may be developing. Hydrometeorology Data Delivery The number of data locations has historically increased by approximately 10% per year and is expected to continue increasing at this rate. With improved efficiency the target through FY08 is to maintain the HADS data acquisition and processing time to 3.5 minutes even with continually increasing data loads. In FY09, the hydrology program is striving to meet an ambitious new level of efficiency by reducing the processing time to 3 minutes even as data loads continue to increase. Schedule Performance The schedule performance represents the percentage of Hydrology Program milestones being accomplished as planned is targeted to increase to 96% in 2010 from the 2005 baseline of 95%.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Academia receives funding through grant applications requiring applicants to identify performance measures that link to program goals. Grants contain specific cost, schedule, and performance requirements that are consistent with the program's strategic outcomes. In addition, grants have very specific deliverables and performance measures that relate to and support the output and outcome goals of the program. Contractors commit to work toward the annual and long-term program goals through contract deliverables or performance criteria for performance-based incentive contracts, which are linked to the program goals. Contractor performance is tracked through monthly status reports and contract deliverables. Contractor commitment for a performance-based contract is reinforced by evaluating established performance criteria for their cost, schedule, and technical performance and granting incentive awards based on the contractor's performance. Commitments from government partners to work toward both annual and long-term goals are accomplished through joint working groups at the national, regional, and local level. MOUs with Federal agencies contain performance requirements that link to the program's goals. The main Federal partners, their roles and responsibilities, and interagency cooperation are shown in the "Federal Hydrology (River and Flood Forecasting) Infrastructure" document provided in the evidence. These Federal partners support the program's overarching goals. Specially, they provide data required to produce forecasts. For example, the U.S. Geological Survey is committed by their mission to "provide water information that benefits the Nation" which includes real time river level information used by the NWS in developing river forecasts. Other government partners such as the U.S. Bureau of Reclamation and U.S. Army Corps of Engineers provide water flow information through their reservoirs. This information is used by the NWS to provide river forecasts which the reservoir operators use in turn for management of their reservoir. In these cases the government partners are committed to providing quality information for achieving their mission as well as contributing towards the goals of the Program.

Evidence: MOUs with other federal agencies; Program Strategic Plan; Contract Statement of Objective (SOO); Award fee structure for Performance Based Contracts; 'Federal Hydrology (River and Flood Forecasting) Infrastructure' document.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: In 2006, the National Research Council (NRC) published a report (see evidence) in response to a request from the Hydrology Program to review and assess the full scope of the AHPS program. Overall, the study "found AHPS to be an ambitious program that promises to provide services and products that are timely and needed." Recommendations from this report identified areas where the management of the AHPS program could be improved. In response to this evaluation, the Program created a group to focus on planning, acquisition, tracking, and resource analyses. The Hydrology Program also chartered 'AHPS Theme Teams', including personnel from headquarters, regions, and local offices, to assist with budget and prioritization decisions. These and other improvements to AHPS program management have led to a transparent and inclusive budgeting and prioritization process which address the concerns raised by this evaluation. Biennial reviews of customer satisfaction and program relevance are conducted for the Program by an independent consulting firm specializing in the American Customer Satisfaction Index (ACSI). The Hydrology Program scored well in 2006, with an overall ACSI of 78 compared to the ACSI public and private industries average of 74.4. Recommendations from the 2006 survey indicate the Program should move forward with flood inundation mapping, provide uncertainty information for short-term forecasts, and support personalized communication and outreach with our customers and partners. Based on these recommendations, the Hydrology Program is continuing activities in flood inundation mapping and accelerating the research-to-operations for uncertainty information for short-term forecasts. Additionally, the Hydrology Program Manager is working with the NWS Corporate Board to emphasize a communication and outreach function at every RFC. In 2005, the Department of Commerce, Office of Inspector General, published an Inspection Report entitled "The Northeast River Forecast Center Is Well Managed, But Some Improvements Are Needed." The report recommended that the NWS develop, document, and implement a timeline and action plan for completing a comprehensive river forecast verification system. In response, a team of headquarters, regional, and field office personnel created the 'National Weather Service River Forecast Verification Plan', 2006. The Hydrology Program has begun operational implementation of verification capabilities and will phase in the comprehensive river forecast verification system over the next five years. In 2002 the National Hydrologic Warning Council published a report entitled "Use and Benefits of the National Weather Service River and Flood Forecasts." This independently-prepared report quantifies the cost effectiveness of the Hydrology Program, projecting that $766 million in annual benefits (indexed to 2000 price levels) could be achieved from the successful implementation of the Hydrology Program's AHPS and Water Resource components.

Evidence: "Toward a New Advanced Hydrologic Prediction Service (AHPS)", National Research Council Report (2006). This NRC committee consisted of members from academia, trade organizations, private sector and partner agencies. Through open and closed sessions the committee evaluated the scientific and technical as well as the programmatic aspects of the AHPS initiative. NOAA/NWS employees, representatives from other related federal agencies, and water resource and emergency management practitioners attended and participated in open sessions. Additionally, the committee split into small groups and visited sites across the country to collect information and better understand how federal, state, and regional agencies use hydrologic forecast data. "The Northeast River Forecast Center Is Well Managed, But Some Improvements Are Needed", U.S. Department of Commerce's Office of Inspector General, Inspection Report No. IPE-17259 (2005). This evaluation was conducted as a part of the periodic evaluation system required for all offices in the Department of Commerce. The purpose of the inspection was to assess the adequacy of the NERFC's programmatic and administrative operations, determine the effectiveness of management and NWS oversight of the NERFC, and assess how the NERFC coordinates its activities with federal, state, and local government agencies and other water management organizations. This inspection was conducted in accordance with the Quality Standards for Inspections issued by the President's Council on Integrity and Efficiency in 1993. The review lasted from December 14, 2004 through February 4, 2005, with an onsite visit from January 3-7, 2005. "Use and Benefits of the National Weather Service River and Flood Forecasts", National Hydrologic Warning Council report (2002). This report quantified the benefits accruing from improved water use and management practices with better forecasts for the specific regions of the United States. The methodology included establishment of an independent review team representing the external hydrologic community. This enabled the NWS to establish priorities for improving hydrologic services and implementing AHPS nationwide. CFI Group's Report "National Weather Service Customer Satisfaction Survey - Hydrologic Services Program - Executive Report 2006". This report summarizes the findings of the 2006 online survey. The NWS provided the survey link to customers and posted the link on various NWS web pages. The independent consulting firm uses the ACSI methodology to identify the causes of customer satisfaction and relates satisfaction to organizational performance measures such as the customer confidence in the services they receive. The ACSI (www.theACSI.org), established in 1994, is a uniform, cross-industry measure of satisfaction with goods and services available to U.S. consumers, including both private and public sectors. "National Weather Service River Forecast Verification Plan", NWS Report (2006)

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Program uses a formalized and standardized planning, programming, budgeting, and execution process to guide research and management activities through an annual review of priorities and budget allocations to ensure the Program is on track to meet its strategic objectives. As part of this standardized process, the Program undergoes an annual strategic planning exercise that looks out 3-8 years and identifies future requirements and trends that will require re-targeting resources. This strategic planning is informed by stakeholder input collected during the year. The guidance from the planning phase informs the creation of the annual budget during the programming and budgeting phases. As the budget moves into the execution phase, an Annual Operating Plan ties all resources to milestones, which in turn are linked to performance measures. The completion of the milestones is tracked throughout the year and reported quarterly. All this information is housed in a web-enabled database system known as the electronic Annual Operating Plan or eAOP, which ensures consistency and accountability. This process helps to guide research and programmatic activities through an annual review of priorities and budget allocations to ensure progress in meeting strategic objectives. NOAA's annual budget requests include the full cost of attaining performance goals and how programmatic increases impacted performance. Budget and performance data are integrated and the impact of funding decisions on performance is shown. The NOAA Budget request identifies the Hydrology Program performance goals (e.g., the economic benefits provided along rivers and lakes due to improved flood mitigation, enhanced water management, increased agriculture, and more efficient hydropower generation resulting from forecast information) and identifies how these performance goals are influenced by different funding strategies. Additionally, the NOAA Budget request identifies how the Hydrology Program goals contribute to NOAA's performance goals to improve flash flood warning lead time and accuracy.

Evidence: NOAA's annual budget requests, Line Office and Program Annual Operating Plans (AOPs) that outline milestones and performance measures that reflect these priorities.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: NOAA's planning, programming, budgeting, and execution process identifies program deficiencies through a review and analysis of mission requirements and the services provided to program partners and customers. External program reviews further identify program requirements (deficiencies), e.g., National Research Council reports and Inspector General reviews. The Hydrology Program has corrected deficiencies identified by external reviews (e.g., providing text and graphical forecast information via the internet rather than text only; uncertainty information; and information for smaller areas rather than only a county size or larger scale) through the provision of (1) forecast information via the internet with certainty information and (2) water resource forecast information for geographic areas smaller than watersheds. These deficiencies are addressed by the program Advanced Hydrologic Prediction Service (AHPS) and the Water Resources activity. AHPS provides river forecasts with uncertainty information and addresses the long-term performance goal, "River and lake sites with advanced hydrologic prediction capability (forecast locations)." AHPS extends the program's basic services of 1, 2, and 3 day forecasts out to weeks and months; provides uncertainty information indicating the likelihood a forecasted event will occur; and delivers information with visual displays over the internet. Prior to AHPS, forecasts were provided as text only information. AHPS' goal is to provide enhanced services at 4,011 forecast locations through the United States. AHPS effectiveness is evidenced with the IG report (Inspection Report No. IPE-17259/August 2005), "The Advanced Hydrology Prediction Service will enable?? [the NWS] to provide both more information and increasingly complex hydrologic information to its users. This is expected to increase the demand for more hydrology products and services and bring in new customers and partners - some with an understanding of hydrology, some without." The Hydrology Program's Water Resources activity began in 2006 and addresses the performance goal, "Nationwide high resolution forecast coverage (percent)." Water resources addresses the provision of water resource forecast information for geographic areas smaller than watersheds (i.e., higher resolution forecasts via distributed modeling). Projects to address this deficiency for the provision of forecasts for smaller areas include: o The release of a high-resolution hydrologic model to NWS River Forecast Center operations, the key modeling component needed for the creation of higher resolution forecasts. o The program will demo (August 2007) the "Community Hydrologic Prediction System (CHPS). CHPS has two significant benefits: 1) using state-of-the-art computer architecture, CHPS will reduce software maintenance demands and facilitate the transition of science advances into operations; and 2) CHPS will facilitate community-shared scientific development projects.

Evidence: Hydrology Program Operating Plan and annual operating plan; NOAA and NWS strategic plans. External program reviews such as National Research Council and Inspector General reviews include: "Toward a New Advanced Hydrologic Prediction Service (AHPS)", National Research Council Report (2006) "The Northeast River Forecast Center Is Well Managed, But Some Improvements Are Needed", U.S. Department of Commerce's Office of Inspector General, Inspection Report No. IPE-17259 (2005)

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The hydrology program regularly collects performance information to manage and improve the program; this includes quarterly measures such as flash-flood forecast accuracy and lead times to evaluate the adequacy of training programs, hydrologic tools and procedures used by forecasters to generate and issue forecasts; and measures such as schedule performance and implementation status to measure timely delivery of enhancements. Managers review these measures during monthly, quarterly and annual reviews to evaluate system performance, and when needed, develop plans to correct or improve the measures for efficient delivery of hydrologic services. The NOAA Local Forecasts and Warnings (LFW) Program is a key partner for delivery of hydrologic information to local communities through its distribution structure of 122 Weather Forecast Offices (WFOs) and 19 Weather Service Offices (WSOs). Performance measures such as Flash Flood Warning (lead time and accuracy) are a joint measure of the Hydrology and LFW Program. These measures are reviewed monthly, quarterly and annually to monitor overall program performance and identify areas for improvement. Measures such as the timely collection and processing of hydromet observation data for delivery to the NWS field offices are essential in providing accurate river and lake forecasts. Review of this measure during development of the Hydrology Program Annual Operating Plan resulted in implementing upgrades for the associated hardware and processing software to reduce the collection and processing time from a target of 8.5 minutes in FY04 to a current target of 3.5 minutes which allow the WFOs and RFCs additional time to produce more accurate forecasts. Biennial Customer Satisfaction Index surveys measure customer and partner satisfaction of program services. The survey is used to evaluate program performance against established targets to assess program performance and identify areas to be improved. A recent initiative to provide water and soil condition forecasts via a national digital database was initiated in part due to customer feedback and evaluations. Key performance information is also used for personnel performance evaluations to reinforce the importance of the measures. The NWS has an online verification database that tracks this information which allows NWS managers to view performance by geographic areas (NWS regions and forecast offices). This is important since the topology and climate conditions in different regions of the country may bias the national performance measure and mask problems that need to be addressed. As an example, mountainous areas have fast moving streams which require different techniques and software to forecast flash floods than a slower moving stream in the Midwest. The IT component of the program uses Earned Value Management for cost, schedule and performance milestones to provide managers with additional tools to evaluate program performance.

Evidence: Quarterly annual operating plan reports; annual performance and accountability reports; monthly Earned Value Management reports and quarterly Operational Analysis reports.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Hydrology project statuses are reported monthly to the NOAA Hydrology Program Manager to ensure that project targets and goals are being attained. The NOAA Hydrology Program manager presents quarterly program updates on cost, schedule and performance to the NOAA Executive panel which oversees the NOAA planning, programming and budgeting process. The NOAA Deputy Undersecretary who heads this panel then meets with the NOAA Administrator on a quarterly basis to review cost, schedule and performance results for all NOAA programs. NOAA senior managers are evaluated on program results achieved, such as the meeting of GPRA measure goals while Project managers have performance plans that include quantifiable performance measures and annual milestones that are tied to the program's performance goals. This appraisal is the basis for adjustments in pay, the granting of awards, and other personnel actions. Contractors are held accountable for performance through deliverables specified in their contracts or the performance criteria for performance based incentive contracts. Contractor performance for a performance based contract is evaluated yearly against established criteria for their cost, schedule, and technical performance. Contractor incentive awards are issued based on a predetermined scale for the level of performance reached. Project managers are responsible for each program component and its performance measures are reported quarterly on cost, schedule, and performance for various program components. Program grantees are held accountable for achieving through the terms of the recipient's award. Grants are adhered to by procedures instituted by the NOAA Grants Office via Grants Online, such as 6-month performance/progress reviews. The Hydrology Program Senior Scientist provides oversight for University Grantee activities to assess their performance and makes cost or schedule decisions for the particular work based on progress and applicability of their research.

Evidence: Department of Commerce Financial Assistance Standard Terms and Conditions, Description of the NOAA Executive Panel and Decision Coordination Office. Contract Statement of Objective (SOO), Award fee structure for Performance Based Contracts, NOAA Hydrology Program Annual Operating Plan.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The Hydrology Program funds are spent for the intended purpose in accordance with the NOAA Business Operations manual, and annual budget submission and appropriation. Program funds are spent for the intended purpose as directed by annual spending plans, which are developed and based on the Congressional appropriations. Funds are allocated through a formalized process that includes the prioritization of Program activities. Monthly spend plans are developed based on the Annual congressional appropriation and in consultation with NOAA and the Hydrology Program Manager. Monthly spending reports are generated using the Commerce Business System (CBS) to monitor actual and planned obligations and to ensure timely expenditure of funds. If a monthly variance from the plan occurs, program managers take the necessary corrective action. Execution of the program's operating budget is tracked on a monthly basis and subjected to quarterly reviews conducted by NOAA management and the NOAA Budget Office. Quarterly reviews provide an in-depth review of planned verses actual obligations, including partner obligations, established at the beginning of the fiscal year. Historically, the Hydrology Program has obligated over 95% of the yearly appropriated funds

Evidence: NOAA Business Operations manual, Annual Spending Plans, Commerce Administrative Management System (CAMS), NOAA Quarterly Reviews.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The Hydrology program has several procedures in place to measure and achieve efficiencies and costs effectiveness. The long-term measure of implementation efficiency for establishing forecast locations with the advanced hydrologic prediction capability is designed to improve the program's productivity. This measure ensures the program focuses on finding the most cost effective mix of government and contractor resources for implementing new forecast services. As an example, the contractor resources are directed toward the background calibration work needed to implement services at new locations. It has been found to be more cost effective to calibrate all forecast locations in a watershed at one time rather than the same number in a variety of watersheds, so contract work is assigned by watershed. Examples of the procedures in place to ensure cost effectiveness include competitive sourcing efforts, cross-program and agency coordination, shared services initiatives, and management structures that empower National Weather Service field (forecast) operation managers to review and comment on program progress and direction. The program uses competitive sourcing to provide the "best value" to the government for cost, schedule and technical performance for the contracted services. Incentive-based contracts are also used to encourage the contractor to improve cost, schedule and technical performance during the contract execution cycle. Performance measures are reviewed by managers during monthly, quarterly and annual reviews to evaluate system performance and when needed develop plans to correct or improve the measures for efficient delivery of hydrologic services. As an example, through technology refresh the delivery of hydrometerologic observation data to NWS field offices has been reduced from a 2004 target of 8.5 minutes to a current measure of 3.5 minutes which enable forecast offices to produce more timely and accurate forecasts.

Evidence: Annual budget submissions and annual performance plans. Contracted task activities are procured through competitive contract procedures and include the use of incentives to reward contractors who produce tangible productivity and efficiency gains. Earned Value Management System (EVMS) reports used to measure the cost, schedule and technical performance of a designated project. Hydrology Program performance measures.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program collaborates effectively with several government and non-governmental organizations, including: U.S. Army Corps of Engineers (USACE), U.S. Bureau of Reclamation, Tennessee Valley Association and other agencies to manage reservoir operations; USACE, U.S. Geological Survey (USGS), National Resource Conservation Service (NRCS) and other agencies for the exchange of river level data, data from river gauge stations. This provides current reservoir and river information used by the hydrology models and forecasters for generating river and lake forecasts. The Program undertook efforts to improve transitioning projects from research to operations in the areas of water resources and river forecasting, in response to a few National Research Council (NRC) reports. One of these actions is establishment of the Operation and Service Improvement Process (OSIP) which sets a gate at the end of each phase of project execution. A formal OSIP review, which includes other program organizations, is held at each gate to assure resources are being effectively used, the program is achieving its intended results, and required documentation and logistics are in place prior to entering the next project phase. Another example is the flash-flood and debris flow (mudslide) early-warning demonstration system established for areas in southern California that are susceptible to mudslides as the result of forest fires. The demonstration uses NWS rain fall information and USGS mudslide forecasting models to generate warnings which are issued by the NWS Weather Forecast Office. The Hydrology Program will continue activities such as these efforts to ensure improved collaboration and coordination with other government organizations, academia and the private sector.

Evidence: Annual interagency meetings are held to coordinate common efforts. The program has representatives on various government-wide oversight and planning committees. For example, the Hydrology Program is an active member of the Federal Advisory Committee on Water Information. The committee represents the interests of water-information users and professionals in advising the Federal government on Federal water-information programs and the effectiveness in meeting the Nation's water-information needs. The Hydrology Program is an active participant on the President's Council on the Environment and Natural Resources (CENR) and its Subcommittee on Water and Water Quality (SWAQ). The program also has MOUs with several agencies including: USGS, NASA, and USDA.

Does the program use strong financial management practices?

Explanation: The NOAA Hydrology Program has received a clean audit opinion and is free of material internal weaknesses reported by auditors. Further, there have been no issues with non-compliance with laws and regulations related to financial management. The Hydrology Program follows prescribed Department of Commerce financial management and accounting policies, procedures, controls and uses the Commerce Business Systems (CBS) for its financial management. CBS is compliant with the Federal Financial Management Improvement Act and permits daily review of financial transactions. Also, other fund accounting tools (e.g. construction work in progress) are used to maintain an accurate financial picture. The Hydrology Program conducts quarterly resource and capital asset reviews to monitor the status of projects, operating plans and planned vs. actual obligation variances. Expenditure reports are reviewed quarterly by NOAA leadership.

Evidence: NOAA annual performance and accountability reports; annual spending plans; Commerce Administrative Management System (CAMS); and examples of monthly, quarterly, and annual cost reviews.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The hydrology program has identified and addressed management deficiencies using several mechanisms including independent bodies such as the National Research Council (NRC), the DOC Inspector General, and independent consultants in addition to internal reviews. To address requests from NWS regional offices and the NRC, an AHPS Review Committee (ARC) comprised of headquarters, regional, and operational facilities was established to alleviate coordination difficulties during implementation of the Advance Hydrologic Prediction Service (AHPS). The ARC conducts monthly teleconferences and meets twice a year to review program progress and address any program issues. To assure operational needs are being addressed, teams have been formed for different program areas such as flash-flood services and short- to long-term forecasts. The teams are comprised of headquarters, regional, and operational personnel to provide a cross section of skills and organizational needs. The teams conduct a yearly in-depth evaluation of project activities and make recommendation to the ARC and Hydrology Program Manager for projects which should be undertaken the following year. This has improved the administration and management of the AHPS project. An Operation and Service Improvement Process (OSIP) has been established which provides a structured management process for program execution from research to operations with gates for management approval prior to proceeding to the next project phase. This improves the transition from research to operations by including management from all program organizations from research to operations to assure the project will meet its intended operational need when implemented. New performance measures (e.g. schedule performance, customer satisfaction index, and national high resolution forecast coverage) have been added in response to a recommendation from the NRC and the program's desire to have additional program measures. This has provided additional information for the Hydrology Program Manager to measure program success and establish program objectives and goals for meeting partner and customer needs. To meet future program demands, the effectiveness of the hydrology organizational structure has been analyzed by an independent consultant. This analysis resulted in realignment of the organizational structure and creation of a Planning, Programming & Coordination (PPC) staff to improve program planning, acquisition, tracking, and resource analyses for the Hydrology Program.

Evidence: The NWS Strategic Plan, PPBES, and NWS corporate board operating procedures include standard business practices to enhance agency management, NRC Report (2006), "Toward a New Advanced Hydrologic Prediction Service (AHPS)," Office of Hydrologic Development organizational analysis study, Terms of Reference for the Advanced Hydrologic Prediction Service Review Committee.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The Hydrology Program is improving its forecasts, warnings, and water resource information through the Advanced Hydrologic Prediction Service (AHPS) and the Water Resources Program. The progress is demonstrated by the program's five long-term performance measures. In addition, external reviews demonstrate the program is achieving its long-term goals (Reference: National Resource Council and Department of Commerce, Office of the Inspector General Reports). However, the full AHPS implementation target was extended by three years, and the efficiency goal could be more ambitious. The program's long-term outcome measures quantify the economic benefits along river and lakes due to flood mitigation, enhanced water management, optimal irrigation scheduling and more efficient hydropower generation; and cross-industry satisfaction with the availability and use of the program's services. These measures indicate the Hydrology Program has added over $300 million in annual economic benefits and savings since FY 2000, and the program's customer satisfaction score of 78 is well above the federal government average of 71 and the industry average of 74 for customer satisfaction. The program's long-term efficiency measure represents the cost to implement each river and lake forecast location with advanced hydrologic prediction capability. Current targets represent a 24% improvement in efficiency from FY 2003 to FY 2014. Based on efficiencies realized to date the targets will be re-evaluated at the end of FY 2007 with the expectation that additional reductions will be occur. AHPS became a Congressionally authorized program in FY 2000 with the target to implement the service by FY 2011 for 4,011 forecast locations throughout the United States. At the end of FY 2006, AHPS is 42% complete; its implementation target has been extended to FY 2014. The program's long-term output measure quantifies the percentage of nationwide high-resolution (small geographic area) forecast coverage. High resolution forecast coverage will provide surface and sub-surface water condition forecasts, including soil moisture, which will lead to reduced losses due to floods, increased economic benefits using water forecasts and information, and improved ecosystem management. This capability directly supports the National Integrated Drought Information System requirement to generate short- and long-term drought forecasts. The program's Water Resources activity, which addresses this measure, received its first funding in the summer of 2006. External Partners have also demonstrated commitment to the Hydrology Program's goals. For example, the United States Geological Survey (USGS), one of the Hydrology Program's partners, is collaborating on a project to detect and forecast debris flows (commonly known as mudslides). Through interagency committees and senior management coordination activities, USGS leadership is aware of the Program's targets, and USGS has committed to the provision of geologic and localized flow information for this project.

Evidence: Performance data are documented in annual budget submissions and accompanying annual performance plans, annual Performance and Accountability Reports, line office strategic plans, line office and program annual operating plans, and monthly performance reports. Other evidence includes: National Resource Council "Fair Weather: Effective Partnerships in Weather and Climate Services" report 2003. National Resource Council "Toward a New Advanced Hydrologic Prediction Service (AHPS)" report 2006. U.S. Department of Commerce, Office of Inspector General "The Northeast River Forecast Center is Well Managed but Some Improvements are Needed" report August 2005 Consultant Report "Benefits of Hydrologic Forecasting" report 1997 National Hydrologic Warning Council "Use and Benefits for the National Weather Service River and Flood Forecast" report 2002 Debris-Flow 1st Year Performance Report

Does the program (including program partners) achieve its annual performance goals?

Explanation: Most of the Hydrology Program's annual performance measures are met to some extent. This is evidenced by the following: Flash Floods (Lead Time & Accuracy) * The Hydrology Program met its FY 2005 and 2006 targets for Flash Flood Warning Lead Time (baselined in 2003); flash-flood warning lead times have increased from 8 minutes in 1987 to 50 minutes in 2006. However, the Hydrology Program missed its FY 2005 and 2006 targets for Flash Flood Warning Accuracy (baselined in 2003). * The installation of NEXRAD Radars between 1994 and 1997 provided a significant boost in flash flood lead time and accuracy. Positive trends since 1997 are largely due to the implementation of the Flash Flood Monitoring and Prediction (FFMP) System and forecaster training. Although dramatic gains were made between 1997 and 2005, improvement will slow until the next generation of science and technologies are operational. Earlier goals were revised downward to obtainable targets within the confines of the current technology regime. The deployment of NEXRAD Super Resolution Radar and annual enhancements to the FFMP System have been pushed back from FY07 to FY08. Hydrometeorology Data Delivery * The Hydrology Program exceeded its FY 2004, 2005 and 2006 targets for "hydromet observation data delivery to NWS field offices" (baselined in 2003), and improved its target for FY 2007 onward. Schedule Performance * The Hydrology Program met its FY 2006 target for its schedule variance metric (baselined in 2005). American Customer Satisfaction Index (ACSI) * The Hydrology Program met the FY 2006 target for its biennial American Customer Satisfaction Index (ACSI) metric (baselined in 2004). Annual Operating Plan Milestones * In FY 2005 and 2006, the program met 100% of its milestones included in the NWS Annual Operating Plan. Economic benefits along rivers and lakes * The Hydrology Program has met or exceeded its economic benefits each year since it was baselined in FY 2000.

Evidence: Performance data are documented in annual budget submissions and accompanying annual performance plans, annual Performance and Accountability Reports, line office strategic plans, line office and program annual operating plans, and monthly performance reports.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has demonstrated efficiencies with its long-term Advanced Hydrologic Prediction Service (AHPS) program. The program has a long-term measure demonstrating program cost efficiencies. The measure represents the cost to implement each river and lake forecast location with advanced hydrologic prediction capability. Current targets represent a 24% improvement in efficiency from FY 2003 to FY 2014. Based on efficiencies realized to date, the targets need to be re-evaluated in order to improve their ambitiousness. The program was more efficient in FY 2004 than in FY 2005 or FY 2006; though in each of those years the program outperformed its targets. In order to foster efficiency, the Hydrology Program makes use of competitive contract sourcing efforts, cross-program and agency coordination, shared services initiatives, and management structures that empower NWS field (forecast) operation managers to review and comment on program progress and direction. The program has significantly improved its method of contracting which has improved both the efficiency in providing contractor support and increased competition to provide the best value to the government. A competitive performance based contract which streamlines acquisition of future services to support the Hydrology Program was awarded in FY03 with a service period of up to 10 years. The Hydrology Program also has Blanket Purchase Agreement (BPA) Contracts available which provide alternative sources to maintain continued competition in order to provide cost effective services. Processes in place to continue to improve NOAA's Hydrology Program effectiveness include its participation in NOAA's Programming, Planning, Budgeting and Execution System process. For this, the program's Office of Hydrologic Development (OHD) has established a Planning Programming Coordination group to manage and monitor the program. This includes the development and reporting through updates to the Hydrology Program Operating Plan, Quarterly and Annual Performance Reporting (including information with GPRA data).

Evidence: Hydrology Program Operating Plan; Quarterly and Annual Performance Reports (including information with GPRA data); FY2006 AHPS Program Plan; and IG Inspection Report No. IPE-17259/August 2005.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There are no known Federal, state, local or private efforts that perform assessments of hydrological forecast data, provide forecasts, or have missions comparable to the Hydrology Program (Reference: Question 1.3). Water information activities of other agencies and the private sector look to NOAA's NWS Hydrology Program for flash-flood, river-level and river-flow forecast information. The NWS hydrologic forecast information is critical to many stakeholders as it indicates when, where, and how much water there will be for emergency operation decisions, water resource management, and water use.

Evidence: There are no known Federal, state, local or private efforts that perform assessments of hydrological forecast data, provide forecasts, or have missions comparable to the Hydrology Program (Reference: Question 1.3).

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: NOAA's Hydrology Program has not had a global evaluation. The program has, however, had several external reviews and quality evaluations for key components of the program. In 2006, the National Research Council published a report entitled "Toward a New Advanced Hydrologic Prediction Service (AHPS)." The committee that performed the review consisted of members from academia, trade organizations, private sector and partner agencies and the review focused on the Hydrology Program's AHPS program. Overall, the study "found AHPS to be an ambitious program that promises to provide services and products that are timely and needed." The National Weather Service (NWS) was commended for presenting clear and well-articulated goals for AHPS, and went so far as to state "developing and implementing AHPS should continue to be a high priority for NWS." In 2005, the Department of Commerce, Office of Inspector General, published an Inspection Report entitled "The Northeast River Forecast Center Is Well Managed, But Some Improvements Are Needed." This review focused on one of the Hydrology Program's 13 River Forecast Centers, and included recommendations requiring national support and coordination. Overall, the IG "found NERFC to be a well-functioning office." In 2003, the National Research Council published a report entitled "Fair Weather: Effective Partnerships in Weather and Climate Services." The 238-page report focused on the relationship between academia, the private sector, and the National Weather Service (which includes all of the Hydrology Program). The report states that "the three-sector system has led to an extensive and flourishing set of weather services that are of great benefit to the U.S. public and to major sections of the U.S. economy." The Hydrology Program's 2006 American Customer Satisfaction Index (ACSI) survey, independently performed by Claes Fornell International Group, also indicates strong performance. The Hydrology Program's overall customer satisfaction score is 78, which is an improvement from the 2004 survey (77), and better than the ACSI average, which includes all public and private industries measured (74.4).

Evidence: "Toward a New Advanced Hydrologic Prediction Service (AHPS)", National Research Council Report (2006) "Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts", National Research Council Report (2006) "The Northeast River Forecast Center Is Well Managed, But Some Improvements Are Needed", U.S. Department of Commerce's Office of Inspector General, Inspection Report No. IPE-17259 (2005) "Fair Weather: Effective Partnerships in Weather and Climate Services", National Research Council Report (2003) CFI Group's Report "National Weather Service Customer Satisfaction Survey - Hydrologic Services Program - Executive Report 2006". 1,668 customers completed the 2006 survey. The questions cover a entire range of the Hydrology Program's services.