The use of TOVS level 1-b radiances and increased vertical diffusion

Anthony P. McNally (visiting scientist at NCEP)

European Centre for Medium Range Weather Forecasts
Shinfield Park, Reading, RG2-9AX, UK.

John C. Derber

National Centers for Environmental Prediction

W/NP23, World Weather Building,
Washington DC 20233, USA

Wan-shu Wu, Bert B. Katz
General Sciences Corporation
Laurel, Maryland, USA

ABSTRACT Two changes have been made to the NCEP global analysis and forecast system: i) the introduction of TOVS 1-b radiances into the global analysis and ii) an increase in the vertical diffusion in the free atmosphere. The TOVS 1-b radiance data have not undergone the pre-processing that is usually applied to make the radiances suitable for use in linear retrieval algorithms and direct assimilation schemes. The increase in vertical diffusion was applied to help prevent the generation of spurious tropical lows.

1.0 Introduction

Two significant changes made to the NCEP global analysis and forecast system in January of 1998 are described in this technical procedures bulletin. The first, implemented 13 Jan 1200 UTC, is a change in the use of the TOVS sounding information in the analysis system; the second (14 Jan 1800 UTC) is the increase in free-atmosphere vertical diffusion in the forecast model.

Observations from the National Oceanic and Atmospheric Administration (NOAA) series of polar orbiting satellites have been used extensively in Numerical Weather Prediction (NWP) since the program started with the launch of TIROS-N (Television Infra-Red Observation Satellite) in 1979. The primary instrument carried on this and all subsequent spacecraft in the series is the TOVS (TIROS-N Operational Vertical Sounder), which measures the multi-spectral radiances at the top of the atmosphere in both the infra-red and microwave regions of the spectrum (Smith et al. 1979). The radiances observed depend on the atmospheric and surface temperature structure and composition of absorbing constituents (such as water vapor and clouds) and may thus be used to extract information on these quantities for use in NWP. Section 2 describes the current NESDIS preprocessing of the TOVS data

Recently, operational NWP centers such as the European Centre for Medium Range Weather Forecasts (ECMWF) and the National Centers for Environmental Prediction (NCEP) have moved towards variational analysis methods (Courtier et al. 1993, Parrish and Derber 1992) for use of the information in the satellite data. This allows the direct assimilation of radiance data in the analysis (i.e. without the need for an explicit retrieval step) and has shown a significant positive impact (Andersson et al. 1996, Derber and Wu, 1998). These schemes use radiances from the NESDIS 120-km TOVS data set which is designed to provide the input to the NESDIS operational linear retrieval algorithm. In order to use the radiances in the retrieval, the data are pre-processed to remove the strongly nonlinear effects of cloud, scan geometry and surface emissivity. Routine monitoring of the quality of the NESDIS pre-processed radiances suggests that the pre-processing steps can introduce significant systematic and random errors into the data . Also, some of the pre-processing errors (e.g. originating from poor cloud detection) are likely to be correlated between different channels and in the horizontal, making an accurate specification of radiance observation error characteristics difficult in the analysis. Since, in principle, none of this pre-processing is required for the direct use of radiances in variational analysis schemes, there is a strong incentive to use the unprocessed radiances (traditionally referred to as level-1b data), which should have smaller and simpler error structures. Section 3 describes the methodology for using level-1b data within the NCEP Spectral Statistical Interpolation analysis scheme, and section 4 presents the results of experiments to test the effectiveness of the new scheme in terms of analysis impact and the quality of subsequent forecasts.

Since the beginning of the hurricane season in 1997, NHC hurricane specialists have noted that the AVN/MRF system generates spurious tropical storms. Our diagnostic efforts showed that the model tropics tended to be noisier than analyses and forecasts from other centers. When the boundary layer turbulence parameterization with a new PBL scheme in 1995 was introduced, the free atmosphere vertical diffusion was also reduced by about an order of magnitude from the previous version in stable situations. This allowed the smaller scale perturbations to exist and amplify in the model. Unfortunately, it also allowed computational noise to amplify. We found out that the convection parameterization in the model was producing too much cooling and moistening in the cloud base region. This led to supersaturated layers in a conditionally unstable region, which led in turn to grid-scale precipitation that created a low-level feedback problem. Section 5 describes the changes directed towards the elimination of the spurious tropical lows and shows some of the results. In Section 6 we present some results from the testing at full resolution (T126) of all of the above changes together, i.e., assimilation with TOVS level 1-b data and the increased vertical diffusion. In the final section, a few conclusions and future directions are presented.

2.0 Current pre-processing of TOVS radiance data

The current NESDIS operational processing of the radiance data performs five major adjustments to the data: limb adjustment, microwave surface emissivity adjustment, remapping of the microwave and SSU observations to the HIRS field of views, atmospheric correction of the HIRS window channel and cloud detection and clearing. Since the implementation described in this TPB, NESDIS has made operational a new processing system (RTOVS) which changes many of these procedures and eliminates the microwave surface emissivity adjustment.

As a sounding instrument scans away from the nadir view, it looks through a longer atmospheric path with an associated increase in absorption. This enhanced attenuation essentially causes the altitude at which the channel is most sensitive (or the peak of its weighting function) to be higher in the atmosphere. For tropospheric channels under normal lapse rate conditions (i.e. not inversions) this results in a reduction in the observed radiance relative to that obtained at nadir, otherwise known as a limb-cooling. For channels sensitive to the lower stratosphere, an opposite limb-warming may be observed due to the often reversed lapse rates. This nonlinear variation in radiance with scan position is removed from the radiance data before use in the NESDIS retrieval by a process known as limb-adjustment. This is done using a statistical regression relationship between radiances obtained from off-nadir positions with nadir radiances under the same atmospheric conditions.

Routine monitoring of TOVS radiance data has been done by many NWP centers, comparing measurements with values computed (using a radiative transfer model) from a short-range forecast profile of temperature and humidity at the observation location. These radiance departure statistics are a combination of three different sources of error: i) errors in the observed data (combined with errors due to any pre-processing of the data); ii) errors in the short-range forecast profile; and iii) errors in the radiative transfer model used to compute radiances from the forecast. It is extremely difficult to separate and therefore make any statements regarding errors in any of the individual components. However, it is reasonable to assume that errors due to (ii) or (iii) should not depend on scan position for limb-corrected (i.e. nadir) data, so that any scan dependence of the departure statistics must originate in (i). Fig. 1 shows the systematic component of the departure statistics plotted as a function of HIRS scan position grouped to the nearest TOVS mini-box (there are 18 across any one scan line). It can be seen there is a significant residual scan variation for some channels and a sharp discontinuity near the scan edge (the latter is known to originate from a change in the grouping of limb adjustment coefficients for these positions). These statistics suggest that, even after limb adjustment, the observed data are not flat across the scan line and a scan bias correction must be employed to correct for this residual limb effect.

The surface emissivity in the microwave part of the spectrum varies significantly for different surface types (e.g. sea, ice, land and snow etc.) and for different viewing angles. This can cause strongly non-linear changes in the observed radiance that must be removed before the retrieval. In practice, this emissivity adjustment is applied to MSU channels 2, 3 and 4, but not to MSU channel 1 which is extremely sensitive to the surface. This channel is used as a primary predictor, done in the same way and simultaneously with the limb adjustment process described above (i.e. by regression).

The TOVS actually consists of three different instruments, namely the High-resolution Infra-Red Sounder (HIRS), the Microwave Sounding Unit (MSU) and the Stratospheric Sounding Unit (SSU). The three instruments have different resolutions (i.e. footprint size) and scan geometry and must be re-mapped on to a consistent grid in order to use all of the combined channel information at a single location in the NESDIS retrieval process. This is currently done by identifying the MSU data that surround each HIRS field of view and performing a bi-linear interpolation to produce values of the MSU channels at the HIRS locations.

HIRS channel 8 is in the 11.1-micron window, but has some sensitivity to (i.e. is attenuated by) the presence of water vapor. The magnitude of this absorption is estimated and the measurement adjusted statistically to be an estimate of the radiance that would have been observed in the absence of the water vapor.

The infra-red HIRS radiances are highly sensitive to the presence of cloud in the field of view. A number of tests are employed sequentially to detect when cloud is present, since failure to do so would result in a complete misinterpretation of the data in the retrieval process. These checks are mostly based on the intercomparison of measurements in different HIRS channels with each other and with channels from the MSU. This is done to detect when statistical relationships that were derived and hold in clear air conditions are violated (due to the presence of cloud). Absolute thresholds on the measured radiance in each channel are also applied (see McMillin and Dean 1982 for more details). In most conditions the detection of cloud results in the rejection of the HIRS data (apart from the two uppermost sounding channels that are considered above the cloud influence). Sometimes equivalent clear radiance values are reconstructed in cloudy locations using the so called N-star method (Smith 1968). This infers information about the cloud characteristics using data from adjacent fields of view (under the assumption of a consistent cloud height) and estimates the radiances that would have been obtained in the absence of the cloud.

3.0 Methodology of Level-1b Radiance Assimilation System

3.1 The SSI analysis system

The theory and methodology of the SSI analysis system is described in Parrish and Derber (1992) and Derber et al. (1991) with the particular aspects of TOVS radiance assimilation dealt with in Derber and Wu (1998). A brief overview of the scheme will be given here, but the reader is referred to these publications for more details. The analysis follows the variational approach to data assimilation (described by a number of authors e.g., Lorenc 1986) in which an objective function (often known as a cost or penalty function) represented by

The combined radiance observation error covariance [O+F] has been estimated using statistics of observed minus forecast radiance departures. These provide an upper bound on the true radiance error (inevitably containing a contribution from errors in the 6-hour forecast), which has been reduced by empirical tuning.

For radiance observations the operator Y(X) must compute radiances from the model variables (i.e. temperature and humidity) and should involve a fully nonlinear integration of the radiative transfer equation at each iteration. In practice, the operator is linearized about the background state X_b to reduce the computational expense of the forward operator and improve the rate of convergence of the minimization algorithm (note that the objective function is quadratic, if the forward operator is linear and the error covariances are gaussian). Currently, the operational radiative transfer model is RTTOV described in Eyre (1991). If K(X_b) is a matrix containing the jacobian of the full nonlinear K operator in (1), the linearized forward operator is given by the expression

A significant change to the forward operator for the assimilation of level-1b TOVS data (which have not been limb-adjusted) is that the radiative transfer calculation must be performed at the appropriate scan angle. Comparing observed radiances with computed values suggests that the RTTOV model has a significant scan-dependent bias for some channels (Fig. 2). While this is extremely useful information for the improvement of such models, it is an important source of systematic error (that must be corrected) and is dealt with further in section 3.5 . Another change to the forward operator for level-1b microwave data (which have not been emissivity adjusted) is that non-unit surface emissivity (and therefore, reflected radiation) must be explicitly handled (see section 3.4).

3.2 Pre-selection of TOVS radiance data

Pre-processed radiance data are provided by NESDIS at a resolution of approximately 120 km on about every third HIRS line and spot. The raw level-1b data are at full instrument resolution which for HIRS data from two satellites corresponds to nearly half a million observations in a six-hour analysis period. This is simply too much for the current computational resources available for assimilation and thus the HIRS data volume must be reduced before reaching the analysis. In this first configuration of the level-1b system, infra-red data are only used in clear-sky conditions (which are ultimately to be identified inside the analysis). Therefore the HIRS data are pre-screened to reject the most severely cloud-contaminated observations. This is done using a clear-sky linear regression relationship developed by NESDIS to predict a pseudo-MSU channel 2 radiance from HIRS channel 13, 14 and 15 measured radiances (having weighting functions that peak in similar regions of the atmosphere). If the HIRS data are significantly attenuated by the presence of cloud, the predicted MSU-2 value will be much colder than the value of MSU-2 that is actually observed at the same location (which is insensitive to the presence of cloud in the absence of strong precipitation). This provides a useful mechanism to detect cloud in advance of the analysis and is only used to reject the most cloud- contaminated data. A loose threshold is applied rather than a more stringent test for a number of reasons. First, we do not wish to pass on to the analysis a population of data that closely fits a rather simple linear regression relation (possibly only fortuitously due to compensating effects) that may itself be inaccurate in certain conditions. Second, the colocation (by interpolation) of the MSU-2 data to the HIRS locations can introduce significant errors in certain (e.g. heterogeneous surface) conditions. Third, the MSU-2 weighting function peaks around 700 hPa, which makes the test insensitive to the presence of low cloud. Thus a large number of moderately cloud- contaminated HIRS data are passed on to the analysis that will then be subjected to significantly more stringent screening inside the analysis. It should be stressed at this point that MSU-2 data are colocated to the HIRS data locations purely for the purposes of the pre-screening and are then discarded (i.e. the interpolated MSU-2 data are not assimilated).

3.3 Cloud detection inside the analysis

The HIRS data that survive the pre-screening are passed into the SSI analysis where they undergo a series of quality control steps before being included in the calculation of the objective function in equation (1). Many of these steps are identical to those applied to the assimilation of NESDIS pre-processed radiance data documented in Derber and Wu (1998) which were designed to detect bad data (or pre-processing), but also conditions when the background was unlikely to be representative of the atmosphere observed by the satellite (e.g. over high or mixed topography). Extra checks have to be applied to the level-1b radiances to reject locations where cloud contaminates the infra-red measurements. The first and most powerful check uses observations in the window channel HIRS-8. These are compared with equivalent radiance values computed from the forecast backgroung and the HIRS data are rejected if the observation is colder than the background value by an amount in excess of a pre-determined threshold. The computed value will be a clear-sky integration of the radiative transfer equation that primarily reflects the background skin temperature and (to a much smaller extent) the presence of low-level water vapor. Any significant cold departure of the observation from this value indicates the presence of cloud in the observation. The value of the cold threshold is obviously extremely important and has been set to a very stringent -1.0K. This value was set from an extensive investigation of how well the observed HIRS-8 could be predicted from the background in regions that could be declared cloud-free with a high degree of certainty. Coincident infra-red and visible imagery from the Advanced Very High Resolution Radiometer (AVHRR) and nearly coincident imagery from geostationary spacecraft (the European METEOSAT, American GOES and Japanese GMS) was used to identify cloud-free conditions.

There is a danger that such a stringent test may prejudice the selected population towards locations where the forecast skin temperature is most accurate (or too cold) and locations where the model moisture is most accurate (or too moist). The jacobian of the forward model suggests that a skin temperature change of 2.5K (in the tropics) and 1.9K (at high latitudes) could give a 1.0K change in HIRS channel 8. It can be argued that over-sea errors of this magnitude are infrequent, but this is not so over land or sea ice. In these conditions, forecast skin temperatures alone can be extremely large and the test could allow significantly cloud-contaminated HIRS data to pass the quality control. For example an HIRS-8 observation depleted by the presence of cloud could pass the -1.0K threshold if the forecast skin temperature had a sufficiently large cold error that made the computed HIRS-8 radiance agree exactly with the cold observed value. Of course, no analysis increment would result from HIRS-8 in these circumstances, but for other channels sensitive to the cloud, but less sensitive to the skin temperature there may not be an exact compensation and erroneous increments could result in the analysis. For this reason tropospheric HIRS data are not used over land or sea ice since it is difficult to have confidence in our ability to detect and reject cloud in all conditions.

Another weakness of the HIRS-8 check is that it is insensitive to very thin cirrus cloud when a significant amount of radiation penetrates the cloud and reaches the satellite (i.e. resulting in no significant attenuation of the HIRS-8 observation). To detect these cirrus clouds (which may impact other higher-peaking channels), a threshold check is applied to the departure of the observed HIRS-12 radiance from the forecast computed value. This channel is sensitive to upper tropospheric moisture and strongly attenuated in the presence of cirrus cloud (with the signal from the cloud often in excess of -20K). The potentially large upper-tropospheric moisture errors in the forecast background and the uncertainty in the quality of the radiative transfer for this model for this channel suggest that a very relaxed threshold of -10.0K would be appropriate.

3.4 A forward operator for level-1b MSU data

In order to simulate level-1b microwave radiances (which have not been emissivity- or limb-corrected) from the forecast background, an angle-dependent value of microwave surface emissivity is required. The forecast model does not provide such a quantity as a prognostic variable and so it must be diagnosed as far as possible from other model variables and any other prior knowledge of the surface characteristics.

The surface emissivity model used over ice-free sea requires input parameters of incidence angle, frequency, sea surface temperature (SST) and surface wind speed. It computes the emissivity of a flat sea surface (with dielectric properties determined by salinity and SST) and modifies the calm value with an empirical roughness factor that is a function of surface wind speed. It also estimates further modifications to the emissivity due to wind-induced foam on the sea surface.

The specification of microwave surface emissivity over ice is very difficult due to the many possible variations in the ice state. Field studies have demonstrated that factors such as the age of the ice are extremely important, as is the presence of a mixed-phase surface due to melting (e.g. in the polar summer). At present there is insufficient information within the forecast model to determine the age of the ice and a value of 0.9 has been used (for all angles and frequencies). This is an approximate compromise lying between the value appropriate to new ice (~0.95) and that for multi-year ice (~0.75). In the near future the ice model recently implemented at NCEP will be used to provide considerably more information to specify the microwave surface emissivity; but until then the treatment of ice surfaces must be considered a weakness of the scheme.

The treatment of land surfaces is another difficult area due to considerable variability in the emissivity properties because of the presence of soil moisture, snow (of many different types) and vegetation. This information is carried to some extent by the forecast model, but for implementation of the scheme a fixed value of 1.0 is used. It is planned to use the land surface information provided by the model more in the near future, but this will require an improved understanding of the diurnal, seasonal and horizontal scale characteristics of the parameters available.

With the known weaknesses of the surface emissivity model (particularly over the scattering ice surfaces and land) an effective quality control mechanism is crucial. The approach used here is based on the accuracy to which the measured value of the window channel MSU-1 can simulated from the forecast background. This channel is extremely sensitive to the surface skin temperature and emissivity such that changes in either produce significant changes in radiance. When there is a large discrepancy between the computed and observed value of MSU-1 we may deduce that one or more of the following is true: i) the forecast model skin temperature is very wrong (unlikely over sea but very possible over land and ice); ii) the assumed value of surface emissivity is very wrong or iii) the measurement itself is bad or contaminated by some effect not modeled in the radiative transfer (such as precipitation or high amounts of cloud liquid water). In any one of these instances, it is considered unwise to use MSU-2, which has a small but significant sensitivity to the surface. If the departure in MSU-1 is less than a prescribed threshold of 5K the assimilation of MSU-2 procedes using the predefined background emissivity values as essentially fixed parameters of the forward model (note that the use of MSU channels 3 and 4 is not affected by this test). A particularly useful aspect of the quality control is its ability to detect and reject situations when the underlying surface is not homogeneous (e.g. coastal and ice edge regions). This is a common situation due to the large size of the MSU footprint, which is in excess of 300 km at the edge of the scan (and just over 100 km at nadir).

3.5 Bias correction scheme

The presence of systematic errors in the radiance data or in the radiative transfer model used to compute radiances from the forecast can have a very harmful effect upon the assimilation system. With level-1b radiance data we no longer have to deal with the biases introduced due to errors in the radiance pre-processing that were discussed in section 2. However, spectroscopy errors in the radiative transfer model (and errors in the fast RTTOV model) may have a very different effect applied to non-nadir atmospheric paths. Also, we may expect new sources of systematic error in the microwave channels due to the surface emissivity model and problems in the fast model treatment of reflected microwave radiation (over non-black surfaces).

In Fig. 1, statistics of pre-processed TOVS radiances compared with values computed from the 6- hour forecast suggest the presence of significant residual scan-dependent biases (due to limb- correction errors). Similar statistics have been generated for the level-1b data and the results are shown in Fig. 2 for NOAA-14 data. It can be seen that there is a significant scan dependence of the mean differences for some channels. The most likely interpretation of this is the inability of the radiative transfer model to reproduce the natural limb effect of the observed data (and badly modeled angular variations of surface emissivity). It has been found that for most of the channels showing a large scan variation, the bias is smooth and globally consistent such that it can be effectively removed with a single scan bias correction fixed for each channel. Two channels, HIRS-15 and MSU-4, do show a larger geographic variations in scan bias that is not understood. For simplicity they have been corrected the same way as the other channels, but the radiance errors assigned have been inflated slightly.

The estimation of these biases is a more difficult problem in the absence of any unbiased truth. In the past it has been assumed that the 6-hour forecast field is unbiased (under certain circumstances such as near radiosonde locations) and that any systematic component of observed-minus-background radiance departure statistics is the combined bias of the data and forward model to be removed (i.e. the same as the scan-dependent biases). The systematic differences are found to be strongly air-mass dependent and their removal achieved using linear regression coefficients derived within the analysis following the approach of Derber and Wu (1998). In the past, the number and type of predictors have been chosen to remove as much of the observed-minus-analysis bias as possible (i.e. leaving no residual). While this approach is certainly pragmatic and prevents any systematic forcing of the assimilation system by the radiance data, the obvious problem is that the forecast and analysis may be biased. If this is the case, complete correction of the data and forward model as above will prevent the radiance data from removing the forecast bias.

In the absence of any radiance preprocessing, the main source of bias is that due to the errors in the radiative transfer. A fixed error in our spectroscopic knowledge of a particular channel or in the fast model's representation of this information will result in a constant error in the computed transmittance or optical depth. This effectively causes a mean shift in a channel's weighting function that translates into a radiance bias depending on the magnitude of the atmospheric lapse rate (note that no radiance error would result if the atmosphere were isothermal). Thus the geographic variation of radiance bias due to these fixed radiative transfer errors should be explained by the variations in atmospheric lapse rate. There will of course be components of the radiative transfer error that are not fixed (and may be air-mass dependent), but the contribution of these to a radiance bias will again be largest in atmospheres with strongest lapse rates (and smallest in near-isothermal atmospheres). Two measures of lapse rate (one appropriate to the troposphere and one to the lower stratosphere) are used as predictors for the new bias correction scheme. The measure of the tropospheric lapse rate is the difference between the deep-layer mean temperatures consistent with the MSU-2 and MSU-3 weighting functions computed from the forecast at the observation location. The stratospheric measure is the difference between the values for MSU-3 and MSU-4. Any systematic components of the observed minus forecast differences that are not explained by these predictors are left in the data (i.e. will not be removed) and are assumed to be systematic forecast model error.

3.6 General quality control of data

Other than the checks required to detect the presence of cloud and situations where the microwave emissivity model was inappropriate, no additional quality control checks were introduced to deal with level-1b data (beyond those documented in Derber and Wu 1998 for the pre-processed radiances). The most important of these checks removes data when the observed minus forecast departure (after bias correction) in a particular channel exceeds 3 times its standard deviation. This check is still extremely important for channels such as MSU-4 and HIRS channels 2 and 3. These channels have no sensitivity to cloud or the surface, but can occasionally contain very bad data. The origin of these bad data is not fully understood, but seems to be associated with cases when the instrument counts get close to the pre-defined calibration limits during the conversion to radiance.

3.7 Radiance observation weights

Observation-minus-forecast departure statistics (which provide an upper bound on the combined observation and forward model errors) suggest that the weight given to radiance data in the analysis could be slightly increased (consistent with the removal of the pre-processing stages). However, the changes were small and it was considered safer for the first implementation to retain the values previously derived for the pre-processed data. However, it was found that in areas where the cloud detection tests and other quality control checks were passed there was a very high concentration of HIRS observations. This caused large increments in the analysis that were considered too large (on the basis of information from other types of data and a knowledge of the background errors). The reason for this is that the errors implied by the combined radiance error covariance matrix should be correlated (both between nearby observations and between channels of the same sounding). Since the estimation of these correlations is very problematic a more pragmatic approach has been adapted to the problem. The data are arbitrarily thinned before the analysis by selecting only every 5th sounding that passes the previous quality control. It is hoped that a less ad hoc solution to this problem will be found in the future.

4. Analysis Impact Experiments

Parallel assimilation experiments were run at T62 resolution over a two-week period (22 March 97 to 8 April 97) to test the performance of the new SSI analysis using level-1b TOVS data ((henceforth SSI/1B) compared to a corresponding low-resolution version of the current operational analysis system described in Derber and Wu (1998) (henceforth SSI/OP). On the basis of encouraging results from this initial experiment, a T62 parallel suite was started on the 15 July 1997 to run daily and results compared against a daily-running T62 version of the operational analysis. This section describes the most significant results from the T62 evaluation of the data.

4.1 Data coverage

Differences in TOVS data coverage for the SSI/OP and SSI/1B systems are expected due to different real-time data availability (level-1b is available sooner than the pre-processed radiance data) and due to changes in the quality control procedures.

In order for data to be useful in NWP, they must be made available to the analysis by a certain time. For the analysis valid at 0600 UTC observations taken between the hours of 0300 and 0900 UTC are combined, but they must be available at 09:45 UTC. Fig. 3 shows an example of the NESDIS pre-processed TOVS and level-1b TOVS MSU data that made the analysis cut-off. While delays in downloading from the satellite and global collection of the data will affect both sources, it is inevitable that pre-processing of the radiances at NESDIS will delay the data further. It has been found that an occasional orbit is available to the level-1b system when the pre-processed data have been delayed beyond the cut-off time (and are thus of no use). The data cut-off in the case shown in Fig. 3 was so close that additional MSU data in the level 1-b data made the cut-off, but the additional orbit in the HIRS did not.

For HIRS data it is interesting to compare regions where NESDIS flags data as clear with those determined as cloud-free from the SSI/1B system. Fig. 4 shows typical clear data coverage for HIRS channel 10 from SSI/OP and SSI/1B . It can be seen that there is a general agreement as to where the clear regions are, but there are a number of locations where the NESDIS pre-processing has produced clear data when there are none from the SSI/1B (e.g. the area northeast of Madagascar). This suggests the SSI/1B cloud detection within the analysis is generally more stringent than the checks that are performed (during the pre-processing) by NESDIS using the observed data alone. An obvious difference between the two systems is the data coverage over land where the SSI/1B is unable to verify (with any confidence) that any tropospheric HIRS data are cloud-free. However, it should be noted that most of the NESDIS pre-processed clear data are subsequently rejected over land in the SSI/OP analysis (by the quality control).

The microwave data coverage is different since the level-1b radiances are assimilated on the original large MSU footprint. However, it is still interesting to compare regions where MSU-2 data are rejected in SSI/1B due to the MSU-1 check indicating a bad model of the surface characteristics. The MSU-2 data coverage for the two systems is shown in Fig. 5 and as expected the largest changes are over land and ice cover (also coastal regions) where it is currently very difficult to estimate a good background surface emissivity. Over sea there is some loss of data coverage from both systems. These correspond to areas where the NESDIS pre-processing and the MSU-1 check in SSI/1B have detected precipitation or large quantities of cloud liquid water (e.g. the area southwest of Australia). Again it appears that the MSU-1 check in SSI/1B is more stringent than the NESDIS pre-processing checks, rejecting more data in these areas.

4.2 Analysis increments

The different characteristics of the pre-processed and level-1b radiance data together with changes in the approach to quality control and bias correction are expected to produce different analysis increments due to radiances. In a complex assimilation system such as the SSI, it is often very difficult to uniquely identify the cause of a particular analysis increment or indeed compare increments between different systems after the first analysis since the background field changes under the influence of the data. Maps of 500 hPa temperature increments for the first analysis of a parallel SSI/OP and SSI/1B experiment (which start from the same initial background field) are shown in Fig. 6 Increments that are clearly due to conventional data over land are very similar (as expected), and there is some similarity in the increments over sea. However, there are some large differences in the Southern Hemisphere that can only be due to TOVS radiances. The most striking of these are the two large positive increments (in excess of 3K) in the SSI/OP that are not seen in SSI/1B. Further investigation of these indicates that they are due to large positive radiance departures (up to 1.2K) in the pre-processed MSU channel 2 (note that these areas are predominantly cloudy where no HIRS data are used in either system). These data are in an area where the NESDIS precipitation check has rejected some data, but the MSU-1 check in SSI/1B has rejected the nearly all of the MSU-2 radiances. While there can be no independent verification of which is correct, it reasonable suspect that the positive increments in SSI/OP are due to undetected precipitation contaminated MSU-2 data. The negative increment in SSI/1B centered on 60S/90E is again due to MSU-2 data. A closer examination of the data suggests that both SSI/1B and SSI/OP have similar radiance departures in this channel, but the development of a negative increment in the latter is suppressed by the large positive increment to the northwest (believed to be due to undetected precipitation). While the background field in subsequent analyses of the experiments changes (making comparison difficult), positive increments in SSI/OP where SSI/1B has rejected MSU-2 data are a common occurrence.

A large difference in the analysis increments is seen around 100 hPa with the SSI/1B cooling the background by over 3K at both poles and cools many areas of the tropics by 1K. There is also a strong warming of the Southern mid-latitudes. The primary data causing increments at this level are from MSU channel 4. Closer examination of both systems reveils that the signal is in both data sets without bias correction, but that the bias correction in SSI/OP has removed most of the signal as radiance bias. By examining the radiance departures after one week of assimilation, it can be seen that the SSI/OP is no closer to fitting the MSU-4 radiance data. However, the SSI/1B analysis has drawn to the radiance information and removed most of the differences that are believed to be forecast error. Comparisons to radiosondes demonstrates a much closer fit of the SSI/1B analyses and forecasts to the radiosondes at this level.

4.3 Systematic changes to the analysis structure

Zonally averaged temperature fields for a week of analyses (from the March experiment) have been examined for SSI/OP and SSI/1B and are shown in Fig. 7. The largest mean changes occur around 100 hPa, mainly resulting from the MSU channel 4 adjustment described in the previous section. Large mean changes are observed at 250 hPa particularly in the Southern Hemisphere and tropics (of the opposite sign to those at 100 hPa). It is not clear if this is a direct effect due to different usage of the radiance data or an indirect coupling with the changes above via correlations in the background error covariance (the latter should produce changes of opposite sign above and below the tropopause). Lower in the atmosphere much smaller mean differences are seen. There is no strong evidence to suggest that the more stringent rejection of cloud contamination in SSI/1B (resulting in less erroneous cold increments) and rejection of precipitation contamination (causing erroneous warm increments) gives rise to any significant change in the zonal mean temperatures. The mean temperature changes in the upper troposphere and lower stratosphere have associated differences in the zonally averaged wind fields (via changes in the horizontal height gradient). No significant mean changes to the analysis humidity structure have been observed between SSI/OP and SSI/1B.

4.4 Fit of the background field to data

A useful measure of the quality of an assimilation system is the extent to which the background field (i.e. a 6-hour forecast from the previous analysis) fits the newly-observed data. For radiances the SSI/1B assimilation shows a slightly improved fit in a number of channels (particularly lower tropospheric HIRS and MSU channel 2). While this is encouraging, such a statistic is rather artificial and expected because of the generally more stringent quality control procedures present in SSI/1B to detect cloud and precipitation. A more objective measure of improvement is the fit to radiosonde data. Fig. 8 shows the mean and standard deviation of observed minus background differences for temperature and wind. The statistics are broken down into three bands, Northern Hemisphere extratropics, tropics and Southern Hemisphere extratropics.

4.5 Forecast impact experiments

The forecasts from the assimilation were compared to the operational low-resolution (T62) forecasts for a 39-day period. In Fig. 9 the anomaly correlations are shown (higher values are better). In both hemispheres and at all time ranges the forecasts from the SSI/1B assimilation were better. In the southern hemisphere, the forecast improvements were particularly large. This is not unexpected since the satellite data are the dominant data source in the Southern Hemisphere.

5. Spurious tropical storm problem

As mentioned above, our diagnostic efforts linked the problem of excess tropical storms to both the vertical diffusion in the free atmosphere and to a convective feedback problem. When we tested an increase in the stable-layer vertical diffusion, we were able to reduce but not eliminate the tropical noise, since the grid-scale precipitation feedback problem still created too many tropical disturbances. Tests of changes in the convection scheme to further reduce this problem, however, led to a decrease in the skill of the precipitation forecast over North America. Further work is needed to ensure that both the problems of mid-latitude precipitation skill and tropical noise are addressed.

A series of tests was made in July on the diffusion changes, but unfortunately the system that was used for testing diffusion was also being used to test an upgrade to the land-surface parameterization scheme. We therefore could not assess the impact of these changes on the precipitation forecast skill over North America. However for the tropical oceans, work done cooperatively with NHC personnel and communications with forecasters from the Hawaiian regional office assured us that the new system does reduce significantly the false alarm rate. In Fig. 10 we show the improvement in the 850- hPa rms vector error for three-day tropical wind forecasts. The results for the operational (mrf) and parallel (mrx) systems are given for the month of July, 1997. It is clear that the changes resulted in a significant reduction in error. From the generally smoother appearance of tropical circulations in both analysis and forecast, we concluded that the improvement came largely from the diffusion changes. In order to assess the impact of the change on precipitation forecast skill over North America we reran the analysis/forecast system for the month of August 1995. In Fig. 11 the precipitation threat score for the month is shown for the current MRF system (mrf97) and for the current system plus the vertical diffusion change (mon97). Fig. 12 shows the bias for these two runs. It can be seen that the changes in vertical diffusion have negligible impact on precipitation forecasts over North America.

6.0 Evaluation of combined parallel forecast and analysis system

The changes described in this TPB were tested parallel with the current operational system at full (T126) resolution. Unfortunately, the test were not entirely clean, with the parallel test including changes which were made operational Oct. 25 and which are described in Wu et al., 1997. However, since these changes did not have a large impact on operations in the Northern Hemisphere and had a small positive (but significant) impact in the Southern Hemisphere, most of the changes should be attributable to the changes shown here.

Fig. 13 shows the daily anomaly correlations for the parallel (exp) and the operational systems (ctl). There is a positive impact in both hemispheres for all forecast days and wavelengths, with the exception of the longest waves in the Southern Hemisphere (not shown). Results for wind forecasts at 850 and 200 hPa in the tropics (20S to 20N) are given in Fig. 14 and Fig. 15. A significant improvment in accuracy is apparent, especially at 200 mb.

7.0 Summary and Future Work

Two important problems were addressed in this TPB. First, the use of the TOVS satellite data was improved by the switch to TOVS level 1-b data. Second, the problem of spurious lows in the tropics was greatly reduced. Both of these systems showed significant improvement in the forecasts and analyses, with the TOVS level 1-b data showing large positive impact throughout the Southern Hemisphere and above 250hPa over the whole globe. The removal of the spurious lows also resulted in significant improvement in the forecast of the 850hPa winds.

The use of the level 1-b data should result in smaller observational errors in the data, earlier access to the data, the ability to use more than two satellites (since NESDIS is restricted to processing only 2 satellites beyond the 1-b stage), and the possibility of using additional information (e.g., cloud, surface, precipitation) in the data.

Further work with the level 1-b data should result in additional information being extracted from the data. The radiative transfer code has several known weaknesses (especially for the moisture channels and the surface emissivity estimation). Also, the data is not being used at its highest resolution. Further work to incorporate a correlated error and/or to revisit the reduction in resolution of the data should allow the use of more of the observations. Also, the quality control and cloud detection problem over land and ice could allow greater usage of the data in these areas.

The switch to the TOVS level 1-b data is also very significant for future satellite usage. With the launch of NOAA-K in the spring of 1998, it is anticipated that the use of these new data will be greatly facilitated by using the NOAA-K level 1-b data. The level 1-b data should be available for testing at a substantially earlier date than the processed data. Also, with the NOAA-K data there are three different footprints for the sounder rather than two as with the earlier satellites. This is likely to result in larger problems in the processed data from the collocation adjustment.

REFERENCES

Anderssen E., A. Hollingsworth, G Kelly, P. Lonnberg, J Pailleux and Z Zhang, 1991: Global observing system experiments on operational statistical retrievals of satellite sounding data.Mon. Wea. Rev.,,119,1851-1864.

Courtier P. et al, 1993: Variational assimilation at ECMWF. ECMWF Tech. Memo. 194, 84 pp. (Available from European Centre for Medium-Range Weather Forecasts, Sheffield Park, Reading, Berkshires R62 9AX, UK).

Derber J.C. and Wu, W-S, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Accepted for publication in Mon. Wea. Rev.

Derber J.C., D.F. Parrish and S.J. Lord, 1991: The new global operational analysis at the National Meteorological Center. Wea. and Forecasting,6,538-547.

Eyre, J.R., 1991: A fast radiative transfer model for satellite sounding systems. ECMWF Technical Memo 176. (Available from European Centre for Medium-Range Weather Forecasts, Sheffield Park, Reading, Berkshires R62 9AX, UK).

Lorenc, A.C., 1986: Analysis methods for numerical weather prediction. Quart. J. Roy. Meteor. Soc.,112,1177-1194.

McMillin, L.M. and C. Dean, 1982: Evaluation of a new operational technique for producing clear radiances. J. Appl. Meteor.,21,1005-1014.

Parrish, D.F. and J.C. Derber, 1992: The National Meteorological Center's spectral statistical interpolation analysis system. Mon. Wea. Rev.,123,1747-1763.

Smith, W.L.,1968: An improved method for calculating tropospheric temperature and moisture from satellite radiometer measurements. Mon. Wea. Rev.,96,387-396.

Smith, W.L., H.M. Woolf, C.M. Hayden, D.Q. Wark and L.M. McmMillin, 1979: The TIROS-N operational vertical sounder. Bull. Amer. Soc.,60,1177-1187.