Algorithm

There are two different blending algorithms used for TPW over-ocean and over-land. This page contains simple descriptions of the blending algorithms used for the blended TPW products. More details please refer to Kidder and Jones.

1. Over Ocean Algorithm

1.1 Histogram Correction

The first step in the ocean blending algorithm is the construction of histograms of TPW values for a five-day period. A histogram is constructed for each satellite instrument at each scan position. The assumption is that in a five-day period, each scan position of each instrument will sample the global distribution of TPW.

Let n(i_TPW, i_SCAN, i_SAT) be the five-day histogram, where i_TPW is the integral value of the retrieved TPW value in millimeters (range: 0-100), i_SCAN is the scan position, and i_SAT is the index for the satellite (range: 1 to n_sat).

The cumulative probability distribution function is defined as, where PDF ranges from zero to one.

The second step in the process is constructing a reference PDF. While the “true” TPW distribution is not known, one set of observations can be chosen to be the reference and an adjustment can be calculated. When the adjustment is applied to other observations it makes the distribution approximate the reference distribution. Currently the average TPW PDF (scan positions 6–25 only) of the AMSU instrument is used as the reference PDF. Figure1 shows an example of the reference for the five day period as the red line.

Figure 1. Illustration of the blending algorithm. [From K&J Fig. 1.]

The third step in the process is the construction of an adjustment for each scan position of each instrument. The blue line in Fig. 1 shows the cumulative PDF for scan position 32 on the DMSP F14 SSM/I for the time period above. The TPW histograms are tabulated with 1-mm-width bins centered at 0.5 mm, 1.5 mm, etc. For each bin from 5.5 mm to 68.5 mm (the 64 x_i values) a y_i is interpolated such that the observed cumulative PDF has the same value as the cumulative reference PDF. This step is illustrated in Fig. 1 for a subset of the x_i and y_i values. Then, applying the adjustment is just a simple process of selecting the coefficients (as a function of satellite and scan position), using the observed TPW as x, and calculating y:

adjusted TPW = a₀ + a₁TPW + a₂TPW² + a₃TPW³ (2)

Figure 2 shows the adjusted cumulative PDF (dashed line) for the data shown in Fig. 1.

Figure 2. The dashed line shows the results of adjusting the observed TPW data with the blending algorithm. [From K&J Fig. 2.]

1.2 Remapping

Each orbit of satellites is corrected with the histogram matching method and then mapped with a chosen map projection. The base map was chosen to be compatible with a map used at NESDIS/SAB. It is a Mercator projection with 16 km resolution at the equator. The map is centered at the equator and 160° west. It has 1437 rows and 2500 columns. The upper left pixel is at 71°14’52” North, 20°20’38” East, and the lower right pixel is at 71°14’52” South, 19°35’3” East. The cut line is at 20° East, which was chosen to emphasize ocean areas. Each orbit of each satellite is corrected with the histogram matching method and mapped onto the output map.

1.3 Compositing

Satellite data may be composited or blended in a variety of ways depending on the intended use of the blended product. The most common way to blend data is to average them over a specified time period, and another way is to overlay newer data on top of older data, such that only the newest data are displayed. The overlay method of compositing is favored by forecasters because it is the most up-to-date image possible, and is used for the current operational TPW products. The orbits for the 12-hour period prior to runtime are composited by o verlaying newer pixels on top of older pixels.

2. Over Land Algorithm

The blending process over land is different than over ocean because it relies on different data. The microwave data sources that are currently used over water do not have retrievals over land. Two sources of data are currently used over land include: (1) PW (layered and total) retrievals from the GOES Sounders on GOES-West and GOES-East; (2) ground-based GPS TPW retrievals. These data sets are available only over the CONUS, with a few stations in Alaska, Hawaii, Puerto Rico, Mexico and Canada.

The starting point for the over-land TPW is the blended TPW analysis over ocean. The basic plan for the Over-Land algorithm is to “fill the holes” using GPS data (first choice) or GOES PW data (second choice). GPS data are used in preference to the GOES PW data because GPS TPW values are unaffected by clouds, whereas GOES PW values can be retrieved only in cloud-free conditions. The GOES PW data provide a valuable backup in case the GPS data are not available.

A Barnes analysis (Barnes 1964, Koch et al. 1983) of the GPS data is performed. The Barnes analysis approach has several parameters which must be specified, relating to the spatial influence of the data. Table 1 lists the values used in the Barnes analysis of the GPS TPW data. The values chosen are rather liberal (e.g. allowing an analysis up to 300 km from the nearest station) and geared towards a smooth field and high spatial coverage over CONUS. These values could be modified in the future to emphasize for instance increased spatial structure at the expense of high coverage. In regions with only a few stations in the Barnes analysis (e.g. Mexico, southern Canada), a circular feature can be produced in the analysis as only a few stations are available for interpolation and data points exceed the distance criteria in Table 1.

Table 1. Current Barnes analysis parameters for GPS data.
Barnes Analysis Parameter	Value
Minimum number of GPS data points within range for Barnes analysis to proceed	3
1/e decay length	250 km
Maximum number of stations in an analysis	100 stations
Maximum distance of closest station	300 km
Maximum distance of a station to consider (stations further than this not used)	600 km

The GOES Sounder data are mapped directly into the Mercator projection, and then a 3 x 3 grid box expansion is performed, or up to roughly a 48 km² areal coverage. This is strictly to eliminate holes in the analysis from insufficient coverage of the gridded data. In the future, the GOES Sounder results could be remapped or averaged together to eliminate missing values.

In order to augment the ocean-only TPW product, each point in the starting map, which contains missing data after the addition of the microwave-derived TPW is examined. At each point, the TPW value is selected in the following order:

1. The TPW value from the starting map (if present – GPS or GOES does not replace existing TPW).

2. The GPS TPW from the Barnes analysis (if present).

3. The GOES TPW value (if present).

The search for a TPW value ends as soon as one is found, so that the blended TPW values over ocean remind unchanged, and GPS data are selected preferentially over GOES PW values. No averaging or error-weighting of the data is currently performed, although such approaches are worthy of further study.

3. TPW Anomaly (Percentage of Normal)

To give forecasters an idea about how abnormally moist or dry the Blended TPW Product is, the TPW values from the blended product are divided by the weekly mean TPW values from the NVAP Dataset for 1988-1999 (Randel et al. 1996). NVAP is a daily, 1-degree resolution analysis which was created with TPW derived from SSM/I (ocean), NOAA TOVS (ocean and land), and radiosonde (land). This gives “percent of normal.” There are 52 weekly mean fields used as normal from NVAP. No moving mean or 3-1-3 (three days before, the current day, and the next three days) calculation of the weekly mean is currently performed. The percent of normal product was developed at CIRA for research purposes but has proven useful for tracking atmospheric rivers, return flow of moisture from the Gulf of Mexico, and abnormally dry conditions associated with fire danger.

Click here to read more details about the blending algorithm.