USGS Home
Contact USGS
Search USGS

Error Sources, Uncertainty, Limitations, and Uses

Even so, the final change inventory dataset is undoubtedly not without errors, and these errors contribute uncertainty to the information. This uncertainty, in combination with innate characteristics of the input datasets, leads to limitations of the information about topographic change in the United States. The sources of error and uncertainty, and the known limitations of the change inventory, are cited below so that users of the topographic change dataset and any derived information are aware of the associated caveats.

As with any thematic dataset, it would be advantageous to have detailed information about the accuracy of the delineation of topographic change features across the United States. Unfortunately, there is no readily available reference dataset with which to compare the inventory to generate accuracy statistics. Reference data would have to be specifically generated through manual interpretation of remote sensing data for a sample of locations, as has been done for accuracy assessment of land cover maps. In lieu of such an approach, which is outside the scope of this study, the best that can be done currently to describe the uncertainty of the inventory data is to identify the known conditions that contribute to it.

Inherent properties of the primary input elevation datasets lead to less than perfect results when they are processed to detect significant topographic changes. The first return nature of SRTM elevation measurements and the production artifacts in the NED and SRTM data can lead to false detections of topographic change. Artifact, in this sense, is any characteristic of the elevation data that does not reflect an actual topographic feature on the ground. The residual errors in coregistration of the NED and SRTM data can also lead to apparent vertical changes, but in reality these differences are only due to misalignment of the elevation datasets. The filtering steps that are applied after the differencing/thresholding operation effectively remove many problem areas, but the process is not perfect.

A primary contributor of uncertainty is the reliability of the land cover data, the NLCD, that are used extensively in this study. Land cover information is used as a fundamental input for the accuracy assessment and the subsequent calculation and application of the significant elevation difference thresholds. Land cover data are also used to assign a label to each elevation change polygon, and this label is used not only in the filtering process but also in the accumulation of summary statistics. The NLCD certainly is not 100 percent accurate, so any misclassification errors can be transferred to the change inventory. This thematic error propagation can be manifest in several forms: first, in mislabeling of reference control points, which affects the calculation of land cover based vertical accuracy thresholds for the difference grids; and second, in mislabeling locations on the difference grid during thresholding, which results in the wrong threshold being applied and features being either included or excluded incorrectly from the change mask.

The thematic accuracy of NLCD has been previously assessed, and the results vary across the 10 regions in the conterminous United States used for NLCD production. Generally, the classification is less accurate for the more detailed Anderson Level II classes employed by the NLCD but improves markedly when classes are combined into Anderson Level I designations. The overall accuracy for the 10 regions at Anderson Level II ranges from 38 percent to 70 percent, and the range improves to 70 percent to 85 percent at Anderson Level I. Such an improvement in overall accuracy by going to more general land cover classes is common with land cover maps derived from remote sensing data. The improved overall accuracy at Anderson Level I is important because one of the first filter steps applied to candidate elevation change polygons considers the general land cover of the polygon. Some of the filter steps use the NLCD Anderson Level II classes that have a lesser accuracy. For instance, the NLCD mining class is important for labeling and filtering topographic change polygons, but the user’s accuracy ranges from only 22 percent to 67 percent.

Some potential errors in the change inventory due to the use of this mining class that has a lower accuracy are probably avoided by the final filter step, which removes polygons in close proximity to mines if they do not meet the criteria for significant changes in basic terrain parameters. While viewing the results of the elevation differencing and thresholding operations, the match of significant elevation differences and the NLCD mining class appeared to be quite reliable. This general relationship suggests a use of vertical differences from multitemporal elevation data as ancillary data for improving the delineation of surface mining areas in future land cover mapping efforts.

Another source of uncertainty lies in the land cover based accuracy thresholds used to extract areas of significant elevation change. As described above, the NLCD thematic accuracy is variable, so it is reasonable to accept that some of the reference control points had incorrect land cover labels because of the inherent error in the NLCD. Also, although the reference control points are located throughout the conterminous United States, they likely are not completely representative of the full range of terrain conditions (elevation, relief, slope, aspect). High elevations, high relief, and steep slopes are probably underrepresented in the control point dataset. This would not be surprising, as the origin of the dataset is surveyed points collected for mapping and geodetic projects. Points in high elevation, high relief, and steep slope areas are not often included in ground surveys because of the difficulty in accessing such sites. Thus, the combination of the spatially variable thematic error in NLCD and the incomplete representation of the full variety in terrain conditions in the reference control points leads to less than perfect land cover based accuracy thresholds. Consequently, application of the thresholds may allow some areas of elevation differences to incorrectly pass the criteria test, for example areas of dense coniferous forest in the mountains of the Pacific Northwest. Many such areas would subsequently have been eliminated by the filtering process, but that process was not perfect either. A few errors of commission likely remain in the topographic change dataset.

The dates of the source data for the primary input datasets contribute uncertainty to the overall results of the topographic change inventory and analysis. Specifically, the dates among the NED, SRTM data, and NLCD are inconsistent. The NED source dates (from the quadrangle-based DEMs) are highly variable and range from 1931 to 1999, with the average source date being 1966. The NLCD is derived from Landsat data with a nominal source date of 1992, although dates from the early to mid-1990s were used. The SRTM dataset has the most consistent source date, with all the radar data having been collected within 11 days in early 2000. The variability in source dates leads to inconsistent matches of land cover labels and topographic changes. For instance, most topographic changes due to mountaintop mining in eastern Kentucky have an associated land cover transformation (deforestation). Because of the nominal 10-year difference in source dates for the NLCD and SRTM data, locations that had mining initiated after the date of the NLCD source Landsat data do not show the corresponding change in land cover. The finer time intervals used in the land cover status and trends project do capture more of the mining related land cover transformations, but such changes are only mapped over sample block areas. The use of the new version of NLCD based nominally on 2001 Landsat data would have provided a useful match between the SRTM data and land cover information, but the incomplete status of the NLCD precluded its use for this study.

The presence of several large data voids in the SRTM coverage of the conterminous United States potentially contributes a slight bias to national summary statistics of topographic changes. These voids occur where interferometric radar data were not collected. Even though 99.96 percent of the targeted landmass was mapped by SRTM, a small shortage in the availability of instrument duty cycle hours during the mission meant that some areas could not be covered. As a primary objective of the mission was collection of data for a near-global DEM, the decision was made to deliberately have these areas that could not be mapped fall over U.S. territory where equivalent high quality data were already available.

Note the following void areas on the next figure: northern Florida/southern Georgia, North Carolina-South Carolina border, Rhode Island-Massachusetts border, central Iowa, northern Nevada, western Utah, northern Arizona, and far western Texas. Collectively, these areas sum to 40,600 square kilometers, which represents 0.52 percent of the conterminous United States land area. These void areas are contained within the following ecoregions: Southern Coastal Plain, Middle Atlantic Coastal Plain, Southeastern Plains, Northeastern Coastal Zone, Western Corn Belt Plains, Northern Basin and Range, Central Basin and Range, Arizona/New Mexico Plateau, and Chihuahuan Deserts.

These SRTM void areas affect the topographic change inventory in two ways: due to the nonexistence of SRTM data, no SRTM – NED differencing, thresholding, and filtering operations were done for these areas, so there is no delineation of topographic change areas; and the summary totals for any accounting and reporting units that include these areas are potentially slightly biased toward lower totals because the change detection could not be done for the complete units.

The use of only two dates of elevation data, and the lack of consistent source dates in the NED, means that the topographic change inventory is limited to binary change detection (change/no change). A significant elevation difference indicates that a topographic change has taken place sometime between the date of the NED source data and 2000. From the data available, there is no way of determining when the change was initiated or completed, so it is not possible to determine annual rates of change. There are far fewer choices for the dates of available multitemporal elevation data than there are for multitemporal remote sensing images for change detection studies. Multiple time slices of elevation data at a fine interval would be required to accurately determine topographic change rates, similar to the methods used by the land cover trends project to present land cover change rates for the eastern United States.

As is often the case for an analysis done with geospatial data over a broad area, the topographic change inventory completed in this study can be used to target areas where more detailed analysis is warranted. Concentrations of change polygons point out regions where human geomorphic processes are active. For example, a concentration of topographic changes related to expanding urban development appears in coastal southern California. Also, a dense concentration of change polygons occurs in the central Appalachians that are subject to surface coal mining operations. Locations like these could be studied in more detail by using recent high-resolution elevation data. Even the existing SRTM – NED difference grids could be reprocessed with thresholds and filters tailored to local or regional conditions so that a more complete delineation of disturbed areas can be made (compared to the delineation from the more restrictive thresholds used for the national study). Given the extensive set of attributes for each polygon in the change inventory, the features could be categorized into subsets for further analysis. For instance, the features could be grouped into categories based on levels of relative confidence in the correctness of topographic change detection. In this example, polygons that are labeled as mining from the NLCD and have all four critical terrain parameters (elevation, slope, relief, aspect) that exceed significant change limits could be listed as “highly probable” changes, whereas polygons with land cover subject to uncertainty from the first return nature of SRTM data (forest and urban/built-up classes) could be listed as “possible” changes. Such a categorization of changes has been previously used for effective presentation of topographic changes.

An immediate, practical use of the topographic change inventory is to inform the USGS mapping program where updates are needed for basic topographic information of the United States. The National Map provides the basic topographic map information layers for the nation, and the NED serves as the elevation foundation. As this study has shown, there have been some significant topographic changes within the United States, and these areas should be targeted for map updates so that the USGS National Map meets its goal of having the most up-to-date geospatial information for the Nation.

This figure shows the USGS 7.5-minute quadrangle maps that contain topographic change polygons. These quadrangles are the ones that are candidates for updated topographic mapping. In fact, a number of these areas have already been updated with the high-resolution, high-accuracy data that have recently been integrated into the NED.

One of the most active regions in terms of topographic changes is the mountaintop mining area in central Appalachia where many of the anthropogenic transformations are due to mining expansion in the 1990s. Fortunately for National Map users, the NED covering the West Virginia portion of central Appalachia has recently been updated with high quality elevation data derived from photogrammetric data collected in 2003.