CALIPSO Quality Statements
Lidar Level 1B Profile Products
Version Releases: 1.10, 1.11, 1.20, 1.22
|
CALIPSO Quality Statements |
|
This document provides a high-level quality assessment of the level 1B lidar data products, as described in Section 2.1 of the CALIPSO Data Products Catalog (Version 2.3) (PDF). As such, it represents the minimum information needed by scientists and researchers for appropriate and successful use of these data products. We strongly suggest that all authors, researchers, and reviewers of research papers review this document for the latest status before publishing any scientific papers using these data products.
The purpose of these data quality summaries is to inform users of the accuracy of CALIOP data products as determined by the CALIPSO Science Team and Lidar Science Working Group (LSWG). This document is intended to briefly summarize key validation results; provide cautions in those areas where users might easily misinterpret the data; supply links to further information about the data products and the algorithms used to generate them; and offer information about planned algorithm revisions and data improvements.
The CALIOP Level 1B data product contains a half orbit (day or night) of calibrated and geolocated single-shot (highest resolution) lidar profiles, including 532 nm and 1064 nm attenuated backscatter and depolarization ratio at 532 nm. The product released contains data from nominal science mode measurement.
The CALIOP Level 1B product also contains additional data not found in the Level 0 lidar input file, including post processed ephemeris data, celestial data, and converted payload status data. The major categories of lidar Level 1B data are:
To make proper use of the CALIOP Level 1B products, all users must be aware of the uncertainties inherent in the data products. The data quality of each product is summarized briefly below:
During the first several months of the mission, the depolarization gain ratio has proved to be very stable, with values falling consistently between 1.02 and 1.05. The uncertainty in these measurements due to random noise is estimated to be smaller than 1% (see the Depolarization Gain Ratio Uncertainty 532, immediately below). Possible systematic errors have not yet been quantified; however, these are estimated to be small, and thus the measured depolarization gain ratio is considered highly reliable.
For the nighttime portion of an orbit, the 532 nm calibration constant is determined for each 55 km averaged profile (11 frames) by comparing the 532-parallel signals in 30 km to 34 km altitude range to a scattering model derived from molecular and ozone number densities provided by NASA's Global Modeling and Assimilation Office (GMAO). This calculation uses equation 4.7 in Section 4.1.2.1 of the CALIPSO Lidar Level I ATBD. The computed 532 nm calibration constants are then smoothed over an interval of 1485 km using equation 4.8. A constant value of the calibration constant is applied to all single-shot profiles in each 55 km averaging region.
The calibration technique used during the nighttime cannot be used in the daytime portions of the orbits, because the noise associated with solar background signals (i.e., sunlight) degrades the backscatter signal-to-noise ratio (SNR) in the calibration region below usable levels. Therefore, for the daytime portion of the orbit, the calibration constants are derived by interpolating between values derived in the adjacent nighttime portions of the orbits.For nighttime calibrations, the uncertainty due to noise is estimated to be typically smaller than 1%. Additional systematic errors may arise from aerosol contamination of the calibration region (less than a few percent), and from large signal spikes seen frequently in the South Atlantic Anomaly (SAA) and occasionally outside the SAA region.
A stratospheric aerosol model is currently being developed to correct for the aerosol present in the calibration region. Upon completion, this model will be applied to calibration processing for subsequent data releases.
Large noise spikes can be present both in the lidar return signals and in the baseline signals. Baseline signals are determined on-board by calculating the mean signal value over 15000 data points (1000 15-meter samples in the 65 to 80 km altitude region from each of the 15 shots within a frame). This calculation is performed for each frame, and the resulting value is subtracted from each sample of all profiles in that frame. The presence of large outliers -- i.e., "spikes" -- in the backscatter signals in the calibration region tends to bias the calibration constant toward a larger value. On the other hand, the spikes present in the baseline region can cause and erroneous overestimate of the measured baseline signal, and the subsequent subtraction of this baseline value will thus introduce a bias in all data within the frame, causing it to be lower than it otherwise should be. This in turn tends to bias the calibration constant toward a smaller value. Threshold-based data filtering schemes are applied to 532 nm data to remove large spikes in the lidar signal prior to performing the nighttime calibration. Two threshold boundaries - a maximum and a minimum - are set. By excluding values outside this range, large signal excursions are effectively removed. Spikes with smaller magnitudes may remain, depending on the selection of the maximum threshold value. Perturbations to the calibration due to spikes in the baseline region can be only partly eliminated by this kind of threshold-based filtering scheme. However, by properly selecting the threshold limits, the impacts of spikes in the calibration region and the baseline region will cancel each other out to some degree. Preliminary comparisons of CALIOP's 532 nm attenuated backscatter coefficients, which are critically dependent on the accuracy of the calibration, with validation measurements acquired by the LaRC airborne high-spectral-resolution lidar (HSRL) and Goddard's airborne Cloud Physics Lidar (CPL) show consistency to within a few percent.
Because the daytime calibration constants are interpolated from nighttime values, the uncertainties contained in the nighttime calibration are transferred to daytime. Additional error may arise from the selection of interpolation scheme. In general, the uncertainty for daytime calibration constants is somewhat higher than the uncertainty for the nighttime values.For each granule (day or night) a single, constant value (granule mean) for C1064 is derived by averaging all individual calibration constant estimates that were obtained. This granule mean serves as the calibration constant that is subsequently applied to all 1064 nm profiles in the granule.
We note that the procedure used in the 532 nm calibration cannot be applied for the 1064 nm measurements, because the molecular scattering at 1064 nm is ~16 times weaker than at 532 nm, and because the avalanche photodiode (APD) detector used in the 1064 nm channel has significantly higher dark noise than photomultiplier tube (PMTs) used in the 532 nm channels.If a sufficient number of cirrus clouds are present in any granule, the uncertainty due to noise in the granule mean of the 1064 nm calibration constant can be very small. Larger systematic errors may arise from the assumption that the cirrus color ratio (the ratio of backscatter coefficients at 1064 nm and 532 nm) has a constant value of 1.0. A very preliminary study on the ratio of gain and energy-normalized, range-corrected signals (i.e., the quantity X defined in equations 3.7 and 3.8 in the CALIPSO Lidar Level I ATBD (PDF)) at 1064 nm and 532 nm in selected dense cirrus clouds shows a distribution having a width of exceeding 10% of the mean value.
The 532 nm attenuated backscatter coefficients are reported for each laser pulse as an array of 583 elements that have been registered to a constant altitude grid defined by the Lidar Data Altitude field.
Note that to reduce the downlink data volume, an on-board averaging scheme is applied using different horizontal and vertical resolutions for different altitude regimes, as shown in the following table.
| Altitude Range (km) | Bin Number | Horizontal Resolution (km) |
532 nm Vertical Resolution (m) |
1064 nm Vertical Resolution (m) |
|---|---|---|---|---|
| 30.1 to 40.0 | 1-33 | 5 | 300 | N/A |
| 20.2 to 30.1 | 34-88 | 5/3 | 180 | 180 |
| 8.3 to 20.2 | 89-288 | 1 | 60 | 60 |
| -0.5 to 8.3 | 289-578 | 1/3 | 30 | 60 |
| -2.0 to -0.5 | 579-583 | 1/3 | 300 | 300 |
The random error contained in lidar measurements consists of two parts. One is due to the variation in the received laser scattering signal from the atmosphere. The other is due to the variation in the background signal. Both parts have to be taken into account when estimating the random error. The random error arose from the scattering signal can be estimated using the NSF. The random error due to the background signal is the measured RMS noise.
| Bits | Interpretation |
|---|---|
| 1 | 532 nm parallel channel missing |
| 2 | 532 nm perpendicular channel missing |
| 3 | 1064 nm channel missing |
| 4 | Not geolocated |
| 5 | 532 nm below RTP threshold |
| 6 | 1064 nm below RTP threshold |
| 7 | Historical value used for the depolarization gain ratio |
| 8 | Historical calibration constant used, 532 nm parallel channel |
| 9 | Historical calibration constant used, 532 nm perpendicular channel |
| 10 | Historical calibration constant used, 1064 nm channel |
| 11 | Averaged calibration constant used, 532 nm parallel channel |
| 12 | Averaged calibration constant used, 532 nm perpendicular channel |
| Bits | Interpretation |
|---|---|
| 1 | Excessive underflows, 532 nm parallel channel (any/all regions) |
| 2 | Excessive underflows, 532 nm perpendicular parallel channel (any/all regions) |
| 3 | Excessive underflows, 1064 nm channel (any/all regions) |
| 4 | Excessive overflows, 532 nm parallel channel, regions 1, 3, 4, 5, & 6 |
| 5 | Excessive overflows, 532 nm perpendicular parallel channel, regions 1, 3, 4, 5, & 6 |
| 6 | Excessive overflows, 1064 nm channel, regions 1, 3, 4, 5, & 6 |
| 7 | Excessive overflows, 532 nm parallel channel, region 2 |
| 8 | Excessive overflows, 532 nm perpendicular parallel channel, region 2 |
| 9 | Excessive overflows, 1064 nm channel, region 2 |
| 10 | LRE Flags and/or Quality Flags in SAD packet indicate bad data, 532 nm parallel channel |
| 11 | LRE Flags and/or Quality Flags in SAD packet indicate bad data, 532 nm perpendicular channel |
| 12 | LRE Flags and/or Quality Flags in SAD packet indicate bad data, 1064 nm channel |
| 13 | Negative mean signal, 532 nm parallel channel (any/all regions) |
| 14 | Negative mean signal, 532 nm perpendicular parallel channel (any/all regions) |
| 15 | Negative mean signal, 1064 nm channel (any/all regions) |
| 16 | Suspicious offset calculation, 532 nm parallel channel |
| 17 | Suspicious offset calculation, 532 nm perpendicular parallel channel |
| 18 | Suspicious offset calculation, 1064 nm channel |
| 19 | Suspicious mean signal value, 532 nm parallel channel (any/all regions) |
| 20 | Suspicious mean signal value, 532 nm perpendicular parallel channel (any/all regions) |
| 21 | Suspicious mean signal value, 1064 nm channel (any/all regions) |
| 22 | Suspicious signal range, 532 nm parallel channel |
| 23 | Suspicious signal range, 532 nm perpendicular parallel channel |
| 24 | Suspicious signal range, 1064 nm channel |
| 25 | Laser energy low, 532 nm |
| 26 | Laser energy low, 1064 nm |
| Lidar Level 1B Profiles Information Half orbit (Night and Day) geolocated, calibrated Lidar Profiles and Viewing Geometry Products |
|||
|---|---|---|---|
| Release Date | Version | Data Date Range | Maturity Level |
| August 12, 2007 | 1.22 | August 12, 2007 to December 5, 2007 | Provisional |
| March 1, 2007 | 1.20 | March 1, 2007 to August 11, 2007 | Provisional |
| January 6, 2007 | 1.11 | January 6, 2007 to February 28, 2007 | Provisional |
| December 8, 2006 | 1.10 | June 13, 2006 to January 5, 2007 | Provisional |
Beginning with Version 1.22, the lidar altitude array is calculated as a function of the spacecraft off-nadir angle. The off-nadir parameter used during processing is determined by the tilt of the CALIPSO lidar relative to nadir and is limited to 0.3 degrees or 3.0 degrees. In Versions prior to 1.22, the off-nadir parameter was set to a constant 0.3 degrees. The lidar altitude array is stored in the HDF metadata field named "Lidar Data Altitude".
Since the beginning of operations in June 2006, CALIPSO has been operating with the lidar pointed at 0.3 degrees off-nadir (along track in the forward direction) with the exception of November 7-17, 2006 and August 21 to September 7, 2007. During these periods, CALIPSO operated with the lidar pointed at 3.0 degrees off nadir. Beginning November 28, 2007, the off-nadir angle will be permanently changed to 3.0 degrees.
When comparing data acquired at 3.0 degrees and 0.3 degrees, the altitude difference between two adjacent samples is relatively insignificant, but when summed over the entire backscatter region, the difference is significant. The total span at 3.0 degrees is 57 meters less than it is at 0.3 degrees. It is therefore important to retrieve the altitude array stored within the HDF metadata field in order to maintain the correct altitude registration between data acquired at different off-nadir angles.
The method used to register the 532 nm and 1064 nm backscatter coefficients to the lidar altitude array is changed beginning with Version 1.22. In previous versions, altitude registration was performed by interpolating the lidar profile data to the lidar altitude array. Beginning with Version 1.22, the lidar altitude registration is performed without interpolation.
For all versions, the altitudes corresponding to all backscatter samples are re-computed during level 1 processing, making use of post-processed ephemeris data and measured attitude data. Using the recomputed altitudes, the backscatter data are then re-mapped to the original fixed altitude array.
In versions 1.0 through 1.21, the attenuated backscatter value that is placed in a given altitude bin is a linear combination of the two attenuated backscatter values whose recomputed altitudes are on either side of that bin's altitude.
In versions 1.22 and forward, the interpolation has been eliminated and the attenuated backscatter value of the lowest altitude sample is reassigned to the altitude bin most closely matching its recomputed altitude value. The remaining attenuated backscatter coefficients are then assigned sequentially to the fixed altitude array bins from that bin upward. This approach has the virtue of leaving the highest resolution attenuated backscatter values unmodified.
The final step averages the lidar profiles stored at the fixed 30-meter resolution altitude to the downlink resolution.
Two surface parameters are added in Version 1.22. The parameter named Surface_Altitude_Shift contains the altitude difference between the profile specific 30-meter altitude array and the fixed 30-meter altitude array at the array element that includes mean sea level. The units are in kilometers and the values may be positive or negative. The difference is calculated as: Surface_Altitude_Shift = altitude (profile specific 30-meter mean sea level bin) - altitude (fixed 30-meter mean sea level bin).
The parameter named Number_Bins_Shift contains the number of 30 meter bins the profile specific 30-meter array elements are shifted to match the lowest altitude bin of the fixed 30-meter altitude array. The profile specific array elements may be shifted up or down.
Four column reflectance parameters are added in Version 1.22. The parameters named Parallel_Column_Reflectance_532 and Perpendicular_Column_Reflectance_532 contain the 532 nm parallel and perpendicular bi-directional column reflectance values derived from the lidar background measurements, respectively. The parameters named Parallel_Column_Reflectance_Uncertainty_532 and Perpendicular_Column_Reflectance_Uncertainty_532 are the 532 nm parallel and perpendicular column reflectance uncertainties, respectively.
The Global Modeling and Assimilation Office (GMAO) next generation global atmospheric model, Goddard Earth Observing System Model, Version 5 (GEOS-5), was implemented within the CALIPSO Data Processing System on March 1, 2007. The CALIOP Level 1B data products obtained from the GEOS-5 assimilation model include Molecular Number Density, Ozone Number Density, Temperature, and Pressure.
The four GMAO-5 meteorological parameters are used to compute the CALIOP Level 1B data for the 532 nm and 1064 nm calibration constants and their associated uncertainties. The CALIOP Level 1B calibration constant data products are named Calibration Constant 532 and Calibration Constant 1064. The CALIOP Level 1B calibration constant data product uncertainties are named Calibration Constant Uncertainty 532 and Calibration Constant Uncertainty 1064. The CALIOP Level 1B 532 nm total and perpendicular and 1064 nm attenuated backscatter profiles are derived from the calibrated (divided by calibration constant), range-corrected, laser energy normalized, baseline subtracted lidar return signal. Thus, the following CALIOP Level 1B data products are also affected by the GEOS-5 transition: Total Attenuated Backscatter 532, Perpendicular Attenuated Backscatter 532, and Attenuated Backscatter 1064.
Version 1.20 is a provisional data release. A preliminary comparison of the CALIOP Level 1B calibration constants and uncertainties computed using GEOS-4 and 5 was performed and revealed small differences. Uncertainties and possible biases will be documented when the data are validated.
The CALIOP Level 1B data products prior to March 1, 2007 (June 5, 2006 to February 28, 2007) were processed using GEOS-4. These data will be reprocessed using GEOS-5 and are planned to be released in the Fall of 2007.
A software fix was applied to the Lidar Level 1B Profile code to correct a memory allocation error that caused jobs to terminate prematurely. The science algorithm and code did not change. Because this version update has no impact on the data products being generated, the Data Quality Statement for Version 1.11 is the same as for Version 1.10, Initial Release.
Geolocation and altitude registration have been checked and appear to have uncertainties of less than the sampling resolution (333 m for geolocation and 30 m for altitude). Random uncertainties due to noise in the 532-nm parallel channel calibration, 1064-nm channel calibration and polarization gain ratio (PGR) determination have been assessed and are reported. Potential biases in the 532-nm parallel calibration appear to be small but are still under investigation. The calibration of the 532-nm perpendicular channel relative to the 532-nm parallel appears to be quite accurate. Thus, volume depolarization ratios should be quite reliable. Uncertainties in the 1064-nm channel calibration have not been studied as thoroughly. There do not appear to be gross errors in 1064-nm calibration, but biases on the order of 10% are not unexpected.
Obtain a Portable Document Format (PDF) reader: Adobe Acrobat Reader | PDF Reader software list
