This project activity has clearly defined performance requirements as well as fiscal and
temporal constraints. The project team intends to acquire the best supercomputer system
available for the budgeted level of funding and to do so in a way that accommodates the
termination of the use of the current supercomputer in February, 1999. Maintaining the
acquisition schedule described herein, which is designed to support the withdrawal of the
current system from operations at the completion of the existing contract, while at the
same time providing adequate computational resources is the highest priority for the
project team.
In order to meet NCEP's performance requirements, the computational capacity of
proposed Class VIII systems will be measured by carefully reviewing demonstrated
performance on a suite of NCEP benchmark programs. Once the Class VIII system is
installed and operational, the project team will ensure the continuing acceptability of the
Class VIII system and verify that its utilization supports all pertinent operational schedules
and appropriate scientific goals.
Performance monitoring during the life of the system will be continuous. System
availability of not less than 96 per cent has been the historical goal for high-end systems at
NCEP and this system will meet or exceed that requirement.
During the life of the Class VIII system, the project team will conduct annual reviews of
the system in order to consider its performance and to evaluate opportunities for
extending its capabilities. System upgrades must be available within the initial budget
constraints of this acquisition and the review process which will result in decisions on how
best to enhance overall performance will begin within one year of system acceptance.
The Class VIII system must offer sufficient capacity to support the necessary activities of
the Environmental Modeling Center and other legitimate user groups in addition to the
execution of operational numerical models and other mission-critical applications.
The following table, submitted as a portion of the 1997 NOAA Operational IT Plan,
summarizes some pertinent operational performance measures. Those for fiscal years
1999 and beyond are explicitly dependent upon the timely acquisition of the Class VIII
system and so are most appropriate for this Project Agreement.
Performance Measures |
Measure |
FY 97 |
FY 98 |
FY 99 |
FY 00 |
FY 01 |
FY 02 |
FY 03 |
Hurricane Prediction System |
|
|
|
|
|
|
|
24hr Position Accuracy (km) |
145 |
140 |
135 |
130 |
130 |
125 |
125 |
Mesoscale Prediction System |
|
|
|
|
|
|
|
24hr 1" Precipitation Skill Score |
.22 |
.24 |
.26 |
.27 |
.28 |
.29 |
.30 |
Short-Range Ensemble Forecasting System |
|
|
|
|
|
|
|
24hr 1" Precipitation Skill Score |
.19 |
.21 |
.23 |
.24 |
.25 |
.26 |
.27 |
Global Prediction System |
|
|
|
|
|
|
|
24hr Aviation Wind Error (m/s) |
7.0 |
6.5 |
6.0 |
5.6 |
5.3 |
5.0 |
4.8 |
Coupled Ocean-Atmosphere ENSO Forecast Model |
|
|
|
|
|
|
|
6-month lead NINO3.4 SSTa AC score (5yr running average) |
0.65 |
0.65 |
0.70 |
0.70 |
0.75 |
0.75 |
0.80 |
The shaded entries here are those directly dependent upon the availability of the
computational capacity and performance of the Class VIII system.
The statistics used to measure performance in the table above require some explanation.
In the case of the hurricane prediction system, 24-hour position accuracy refers to the
difference between the actual and 24-hour forecast positions of the centers of hurricanes
or other tropical storms as measured in kilometers. The table shows that the amount of
error for such forecasts should be less than 130 kilometers in the year 2000.
Quantitative precipitation forecasting is one of the most demanding forecast problems
routinely addressed by the NWS. The accuracy of such forecasts is measured using skill
scores for the occurrence of precipitation which exceeds a particular threshold value. In
the case of both the mesoscale prediction system and the short-range ensemble forecasting
system, the table reflects a measure of forecast skill against a threshold value of one inch.
The skill score is a statistic derived by comparing forecast and observed areas for the
event of interest, here the occurrence of greater than one inch of precipitation. A perfect
forecast would yield a skill score of 1.00, whereas a forecast that is completely erroneous
would result in a skill score of 0.00. The figures in the table for the mesoscale prediction
system and the short-range ensemble forecasting system indicate an expectation of an
increase in forecast skill of about 25 per cent between 1997 and 2000.
The indicated performance measure for the global prediction system is the mean difference
between the actual and 24-hour forecast of wind velocity at approximately 34,000 feet
above the surface of the earth as measured in meters per second. The table shows that the
average world-wide error of such forecasts should be not higher than 6.0 meters per
second in 1999.
The final measure in the table relates the performance of the coupled model to the
occurrence of El Nino-Southern Oscillation (ENSO) events. This is a subtle forecast
problem and the measure of the degree to which these events can be successfully predicted
is accordingly complex. ENSO events are typically measured by the magnitude of the
departure from climatological norms for sea surface temperatures within a given index
area in the equatorial Pacific ocean. The official index area that NCEP uses to define El
Nino occurrences and within which it measures forecast skill is the region from 120 to 170
degrees west longitude and between 5 degrees north and 5 degrees south latitude. This
region is known as Nino3.4. The indicated performance measure for the coupled ENSO
model is the correlation of the Nino3.4 sea surface temperature anomalies (SSTa)
comparing current observations with forecasts made 6-month prior to those occurrences,
as averaged over a 5-year period. The table indicates the skill for El Nino forecasts with a
6-month lead time for the period from 1997 to 2001 should reach 0.75. That amounts to
approximately a 15 per cent increase in the level of skill of the coupled model according to
this statistical measure.
Execution
Signed |
October 15, 1997 |
______________________________ |
____________________ |
Wayman E. Baker |
Date |
Project Team Leader, Acting Director of NCEP Central Operations |
Signed |
October 15, 1997 |
______________________________ |
____________________ |
Robert S. Winokur |
Date |
Acting Assistant Administrator for Weather Services |
Signed |
October 15, 1997 |
______________________________ |
____________________ |
Ronald D. McPherson |
Date |
Director, National Centers for Environmental Prediction |