![]() |
U.S. Department of Agriculture National Agricultural Statistics Service Research and Development Division |
of the Director |
|
|
Cropland Data Layer Frequently Anticipated Questions
1. What is on the
Cropland CD/DVD?
1. What
satellites
do you use for your Acreage Estimation Program? 1. What
software is used to create the
classifications? What is on the Cropland CD/DVD? Each
CD/DVD contains the
rasterized Cropland data layer of a particular state, categorization
accuracy
statistics by state, ancillary vector Shapefile data layers (e.g.,
Agricultural Statistics Districts, Area Frame Land Use Stratum, county
boundaries, and major roads), a screen show of the project’s
methodology,
ESRI’s ArcReader freeware software, an image metadata document, and an
ArcReader project that bundles up the raster and vector datasets
together for
display in ArcReader. Do I need any special type of GIS or Image Processor to view the CD/DVD? No, you can install ArcReader from the Cropland CD/DVD or you can go to the ESRI website and download a copy at http://www.esri.com/arcreader. ArcReader will allow you to browse both the raster and vector spatial data Cropland datasets. A premade ArcReader Published Map Documents folder with ArcReader projects can be found in the \PublishedMapDocuments folder. Note:
Users who already have ArcGIS installed on their Desktop systems may
consider using the \Mapdocuments forlder to load the premade ArcGIS
.mxd project of each state. If preinstalled ArcGIS users try to install
ArcReader, they will be required to remove ArcGIS from their desktop,
this is not a recommended course of action! If you already have GIS
capability, you should have no trouble importing the datasets. How many years of imagery are available on the CD/DVD? Each
CD/DVD contains one
year of categorized imagery. Space limitations can make it
impossible to
include any additional years. What format are the images published in? The images are published in GEOTIFF and Erdas Imagine .IMG format. The newest CDL products
are projected in Albers Conical Equal Area with a Spheroid of GRS 1980
and a Datum of NAD83. Each state with the exception of What other spatial data layers are available on each CD/DVD? The Agricultural Statistics Districts (ASD), County boundaries, major roads, railroads, hydrography, and the Agency’s Area Frame Land Use Stratification of the chosen state could be included in ESRI’s Shapefile format. All shapefiles except the Area Sampling Frame were accessed from the National Atlas website. An ASD is defined as a contiguous group of counties having relatively similar agricultural characteristics. The ASD’s used by NASS usually divide each state into as many as nine Agricultural Statistics Districts to make data comparison easier. Each district is more homogeneous with respect to agriculture than the state as a whole. They are usually numbered first from west to east then from north to south. The ASD shapefile of the U.S. is available for download. An
explanation of the NASS Land Use Stratification can be found at http://www.nass.usda.gov/research/AFS.htm
What other information or metadata is available on the CD/DVD? Accuracy
statistics such
as percent correct and kappa coefficients are presented statewide in
.html format, regression analysis by crop type in .html format, the
satellite type and dates of
observation, and the sampling and associated Area Frame information.
An
extensive metadata file is also available, which provides extensive
detail
about the CDL images. Can I re-distribute the images, vector data or ArcReader software? Yes, you
are under NO
copyright restrictions with either the NASS Cropland categorized
imagery or ESRI’s
ArcReader. The NASS Cropland categorized imagery is considered
public
domain and FREE to redistribute. However, NASS would appreciate
acknowledgment or credit for the usage of our categorized imagery. Who created the Cropland datasets? Traditionally
the field
preparation and digitizing work were performed in NASS’s Field Offices,
and the remote sensing analysis performed by the SARS of
NASS. However, in 1997 the SARS entered into a data sharing
partnership
with USDA’s Foreign Agricultural Service and USDA’s Farm Services
Agency. The agreement provided access to Landsat 5 coverage in
the States
selected for the project by the Spatial Analysis Research
Section. The
first States covered with the data sharing partnership were For crop
year 2000, the
states of For crop
year 2001, the For crop
year 2002, For crop
year 2004, the
IRS ResourceSat 1 AWiFS sensor was used over The 2002 Mid-Atlantic based Cropland Data Layer product was sponsored in part by a NASA/Raytheon/Synergy Project through Towson University, with the digitizing and image analysis being performed under contract by NASS. The Cropland Data Layer products were based on the 2002 June Agricultural Survey and the Agriculture Coverage Evaluation Survey (ACES) that coincided with the 2002 Agricultural Census. The 2005, Idaho Cropland Data Layer was created with a cooperative partnership between a Utah State University Grad Student, the United Potato Growers of Idaho and NASS. This partnership produced both a Landsat TM and ResourceSat-1 AWiFS classification over the Idaho Snake River Plain. The 2005 Midwestern CDL DVD update, contains new AWiFS classifications and a revised Wisconsin TM based classification. The new AWiFS classifications cover Nebraska and North Dakota. The Wisconsin revision was performed under contract for the Wisconsin State, Bureau of Environmental & Occupational Health and Department of Health and Family Services. The Wisconsin CDL is now a complete statewide classification that is nearly cloud free, additional small acreage crops are identified and the non-ag land uses across the state are better defined. The 2005 Mississippi Delta CDL DVD product contains four states: Arkansas, Louisiana, Mississippi and the Missouri (bootheel). This is truely a unique product where the intensely cultivated Delta Region was classified using regression tree classifier See5.0 available from www.rulequest.com over the 2001 NLCD defined mapping Zone 45 http://www.mrlc.gov/ for the States of Arkansas, Louisiana and Missouri. The Zone 45 classification results from See5.0 were overlaid on top of the Arkansas, Louisiana and Missouri bootheel, resulting in an accurate ag classification and an enhanced non-ag land use classification leveraging results from the 2001 NLCD products at www.mrlc.gov. This new prototype product will serve as a basis for future CDL land use classifications. The traditional pixel based PEDITOR classification covers the remaining parts of these states. Additionally, 2005 AWiFS classifications of these four states are provided as well. The 2006 Midwestern/Pacific Northwest CDL DVD product contains seven States including Illinois, Indiana, Iowa, Nebraska, North Dakota, Washington and Wisconsin. Illinois and Indiana were processed with Peditor. The remaining States were processed using See5 decision tree software. The Mississippi Delta CDL and the remaining Midwestern and Prairie States were processed exclusively with See5 using the FSA Common Land Unit for ground truth. To whom do I address concerns about the datasets? Distribution issues can be directed to the NASS Customer Service Hotline at 1-800-727-9540. Content questions on
the CD/DVD can
be directed to 703-877-8000 or email HQ_RDD_GIB@nass.usda.gov The
datasets were created
as an offshoot from the Acreage Estimation Program. Historically
used for
its statistical methodology and ability to produce acreage estimates in
table/spreadsheet format. Functionality was extended into the GIS
mapping
world. Program functionality now allows for exporting categorized
images
to commercial vendor formats for mosaicking purposes. Once the
images
were stitched together, the next step was to bundle this data together
and
distribute it within the Agency and to selected researchers in the GIS
community. It was necessary to bundle ArcReader for potential
customers
who do not possess a GIS that could display the images. The
output
products and Acreage Estimation Program attracted the partnerships
previously
mentioned. Now NASS is delivering this product via the web @ the Geospatial Data Gateway. How statistically accurate are the classifications? The
classification’s accuracy metadata is published on the CD/DVD in .html
format. NASS reports the input scenes and sensors used, percent
correct and kappa coefficients, statewide regression analysis, the
sampling frame scheme, and the original cover type signatures, all of
which are displayed in .html format. Look in the /Statinfo directory to
find this information. What are the geo-positional errors or spatial accuracies associated with the Cropland categorized images? The categorized images are co-registered to MDA/EarthSat Inc’s ortho-rectified GeoCover Stock Mosaic images using automated block correlation techniques. The block correlation were run against band two of each original raw satellite image and band two of the GeoCover Stock Mosaic. The resulting correlations were applied to each categorized image, and then added to a master image or mosaic using PEDITOR. The MDA/EarthSat images were chosen as they provide the best available large area ortho-rectified images as a basis to register large volume Landsat images with. The GeoCover Stock Mosaics are within 50 meters root mean squared error overall. See MDA/EarthSat’s http://www.mdafederal.com/home website for further details. The AWiFS images purchased beginning in 2005 were all orthorectified by GeoEye, and there is no longer a need to perform ortho corrections onto the raw or processed imagery. A whitepaper published at the 2005 ASPRS conference discusses the accuracy of the AWiFS images. What new products can be anticipated in 2007? New CDL
products for the Midwest and There are many possible uses for the Cropland data layer inside and outside the farming community. It can be leveraged in a GIS to perform spatial queries against other enterprise GIS data layers. The cropland layer could be extracted and used as a mask for other public or private entities so that they could focus solely on their own interests by eliminating all cropland areas, and focus on areas such as; urban sprawl, watershed analysis, de-forestation,. Other examples range from water quality assessments and monitoring of watersheds, agribusiness facility location, agribusiness transportation routing/forecasting, studying crop rotation patterns and migration trends, crop pesticide applications, wildlife habitat monitoring, and crop stress or blight location. These are just some of the possibilities that are available for this data set. The following table lists some additional uses:
The image metadata files are stored in the \statinfo directory and have the following naming conventions cdlmeta_r_st_20xx.htm, where st is the two character state fips code, and the xx is the two digit year. The metadata is created from software produced by the USGS and can be found at http://geology.usgs.gov/tools/metadata/. What satellites do you use for your Acreage Estimation Program? The
Spatial Analysis
Research Section uses the medium resolution type satellites, including;
Landsat 4/5/7, IRS 1C, and IRS-P6 Resourcesat
1. Currently, it is too costly to use higher resolution
satellites to
perform crop acreage estimation over a large area. Do the classifications cover the entire state? Why are you not performing this program across the entire country? While it would be nice to have complete coverage of all crops in all states during the growing season, it is just not fiscally or humanly possible at this time. We are looking in the future to possibly expand coverage into other states, but they must first meet certain criteria. Since the Acreage Estimation Program uses medium resolution Landsat/IRS imagery for analysis, it is necessary to working in States where agricultural production contains large size/acreage fields. States whose terrain or where planting practices only allow for relatively small size/acreage fields, then medium resolution sensors are not a good match, and the results may be disappointing. The probability of cloud cover during the growing season is also a consideration when evaluating new states. It is possible over the next few years, that if this program remains cost effective, and state partnerships are developed, it could expand into twenty states total, as long as those states have rather large field sizes, human resources are available, medium resolution sensors are still available, have low cloud cover probability during the growing season, and continued budgetary support from NASS. Begining
with
crop year 2006, the CDL program will cover all of the NASS speculative
corn and soybean states, because of the newly developed program
efficiencies, the large swath width and rapid repeat times of AWiFS and
the availability and coverage of the FSA Common Land Unit Program. NASS says this is a Cropland Data Layer product, what about the areas that are not agriculturally intensive? NASS
collects
the remote
sensing Acreage Estimation Program’s field level training data during
the
June Agricultural Survey. It is a national survey collected
randomly
on
selected areas of land. The selected areas are targeted toward
cultivated parts
of each state based on its area frame. Our enumerators are given
questionnaires to ask the farmers what, where, when and how much are
they
planting. Our surveys focus on cropland, but the enumerators
record
all
land covers within the sampled area of land whether it is cropland or
not. NASS uses broad land use categories to define land that is
not
under
cultivation, including; non-agricultural, pasture/rangeland, waste,
woods, and
farmstead. NASS defines these non-agricultural land use types
very
broadly, which makes it difficult to precisely know what specific type
of land
use/cover actually is on the ground. For instance, there is no
breakdown
as to the type of woods in a given field/pasture, that’s where the
power
of a GIS could be useful. If a external forestry GIS layer was
overlaid,
the land use can be accurately identified, and the specific cover type
can be
derived from the data layer. SARS is developing proportional
sampling
schemes over the non-ag intensive areas, allowing for the sampling of
the NLCD over these areas. What observation dates does the SARS typically use for Acreage Estimation? SARS
prefers to use two
dates per scene location. Usually the first scene is selected
early in
the growing season April – May, and the other for the mid growing
season
late June through August. However, it is usually very difficult
to obtain
two dates of cloud free imagery for one scene during the growing
season.
Sometimes SARS substitutes a fall scene instead of a spring scene, but
tries to
avoid using a spring and fall scene together. Two observation
dates are
preferred for crop separability purposes, (i.e., bare soil vs. hay vs.
row or
small grain crop) where crops may have similar phenological properties,
but
produce different reflectance values. Also, two dates help
separate the
planted cover, woodland, pasture/rangeland out of the crop mix.
However,
if only one date of imagery is available, then it is preferable to
choose a mid
season date late Jun - early Aug before crop senescence. Do the classifications added to county and State level match the official NASS estimates? The simple answer is no. The reasons are multiple. First, as explained in the Program Methodology section of this CD, NASS adjusts the classified results by using a regression estimator and ground gathered data from farm operators. The farmer reported data is strictly confidential and only available to NASS employees in a secure NASS site location. The regression estimates are only one input to official State and county crop acreage estimates. In addition to regression estimates, NASS staff use the results of farmer reported data from surveys, Farm Service Agency data where available, agri-business data and the Census of Agriculture data every five years (1997, 2002, 2007 etc.). Thus, the official estimate is the single best number that NASS can come up with giving all of the inputs some representation. The one major advantage of the classified data is that it is available at a geographic level well below the county level. However; it
is possible to pixel count and return acreage numbers close to the
official estimate but, pixel counting does not account for cloud
covered areas as well as the inaccuracies associated with a pixel based
classification. What software is used to create the classifications? PEDITOR
was the
image
processing package used by the Spatial Analysis Research Section to
create
classifications, and produce numerical estimates. It is currently
only
used to produce acreage estimates, until a new extimator is developed
in SAS. Peditor was developed during
the early 70’s using Since
2006 the
CDL products were created using a commercial software suite including;
ArcGIS for ground truth editing, Erdas Imagine for imagery preparation
and pre-classification processing, See5 decision tree software for
image classification and eventually SAS for the regression estimator. What platform does PEDITOR run under? PEDITOR runs under the Windows platform, specifically 2000/XP. PEDITOR has run on a variety of platforms in the past, including ILLIAC 4 supercomputers, CRAY supercomputers, Digital Equipment Corp. DECSYS10 and MicroVax mini-computers, IBM and PDP mainframes, the original MS/DOS, and Windows 3.1. However,
expert system
and automated batch procedures for
clustering/classification/mosaicking/estimation MUST be run on a
Windows 2000
or XP device. PEDITOR as it is now constituted will only run
under the
Microsoft Windows operating systems. Substantial changes would be
required to port all of the programs to run under either Linux or
Unix.
The requisite changes would vary from program to program, with some
relatively minor,
while others would be fairly major. Is PEDITOR software currently being maintained? Yes, NASS
is
maintaining
the PEDITOR code with a systems engineer who is continually updating
the
functionality and code corrections when necessary. NASS is
maintaining
the regression estimator for crop year 2007, and will phase out Peditor
entirely in crop year '08. NASS is investigating the integration of
commercial software applications including; ArcGIS, Imagine, See5 and
SAS to meet the needs of the NASS processing methodology. What software was/is used to create the mosaics? This is
now a three part
answer. For part one, Erdas Imagine was solely used to create the
North
Dakota 97/98 mosaics. PEDITOR performed all of image processing
tasks,
e.g., clustering, classifying, accuracy assessment, and regression
estimation
first. Then the images were exported to Imagine, and
mosiaced.
North Dakota 97/98 was precision registered with the assistance of
Ducks
Unlimited, Now for part two of the answer, SARS has used PEDITOR to precision register each CDL image since 1999. PEDITOR used EarthSat’s Landsat GeoCover ortho-rectified stock mosaics as a basis for precision registration, and then stitching all of the scenes up together. PEDITOR is capable of stitching the categorized scenes by prioritizing, cutting on scene edges, or within county boundaries. So if you want to know what observation date/Analysis District that a certain area/county came from you will have to examine the Analysis District map in \CDL\xxyyinfo.bmp, where xx is the state abbreviation, and yy is the two-digit year. PEDITOR was the software of choice to stitch each state together using the GeoCover product as the ortho-base image from 1999 until 2005. Part three is the newest method change. Beginning with the 2006 crop year, all CDL products were produced with a combination commercial software suite of ArcGIS for ground truth editing, ERDAS Imagine for image preparation and image preprocessing, See5 decision tree software for image classification and PEDITOR was still used for acreage estimation. The new method mosaics the images together before classification, as the images are purchased already ortho-rectified and once the classification is run, no further mosaicing is necessary. Can I get a copy of PEDITOR/See5? Send an
email request to HQ_RDD_GIB@nass.usda.gov.
The
software is in the public domain, and is therefore, free.
However, it is
not designed to be a widespread or particularly user friendly package
outside
of the Cropland Data Layer Project of NASS.
See5 is commercial software available from www.rulequest.com. Can I get training or support in PEDITOR? Generally, NASS goes not provide training for PEDITOR. It is generally reserved for the cooperators/partners within the Acreage Estimation/Cropland Data Layer Program. PEDITOR is maintained by the SARS in-house system engineer. If software corrections or enhancements are reported/requested, they will be evaluated as to their merit for inclusion into the program. Routine updates to PEDITOR are not provided unless requested. PEDITOR will be phased out in '08. Why does SARS use PEDITOR rather than commercial software for their image processing needs? PEDITOR software was developed in the early 1970’s to determine the value of satellites in estimating crop acreage and production. Around that same time, many commercial GIS/image processing vendors were first getting started. PEDITOR was developed with the intent of measuring agricultural production and much effort has been contributed to the refinement of the acreage estimator. PEDITOR possesses many strengths, including; the ability to perform accuracy assessment on a large scale over entire fields rapidly, chaining multiple programs, acreage estimation with the regression estimator RESTP, and cost – FREE. At the
present
time commercial applications are being utilized to process the CDL
since '06. The commercial applications are able to meet the statistical
demands of NASS' CDL program. What software is used for processing batch type jobs on XP? PEDITOR
utilizes XLNT
software developed by Advanced Systems Concepts http://www.advsyscon.com/ for
batch job
submission on Windows XP and NT workstations. PEDITOR submits
computationally intensive batch jobs through XLNT such as, Clustering,
Classification, Multi-temporal Scene Overlays, Image Mosaicing, expert
estimator Restp, and Scene Reformats. XLNT allows for multiple
batch jobs
to queue up on the Windows XP devices. XLNT is used because of
its
capability to incorporate program executions and data in the same file,
and
also since it has some programming features in its native script
language such
as testing and jumps. What other remote sensing/GIS publications/reports has SARS released in the public domain? Here's a
list of SARS
Bibliographies, or view the White Papers
from the NASS website. Why does the mosaic have the appearance of scene or stitch lines running throughout the image? In older CDL products, one of the main causes of scene/stitch lines came from different scene observation dates or Analysis Districts lying adjacent to one other. SARS performs spatial analysis (cluster/classify) on each Analysis District separately, and then stitches up the scenes at a later time. SARS usually breaks up Analysis Districts because of cloud coverage and available imagery. Some observation dates may be optimum for crop separation, while others may be pre-planting or post-harvest. SARS selects the best available scenes for inclusion into the project. However, as you cross over from one Analysis District to another, you may notice a dramatic shift in agricultural/cultivation practices or changes in land use/land cover. Again, these differences can be attributed to different observation dates. Keep in mind that any particular Analysis District may be cut or stitched by either scene edge or county boundary. Other artifacts in the observation dates can also play a role in making the scene edges contrast sharply, such as; atmospheric issues dealing with water vapor or dust particles, and the presence of soil moisture/standing water to name a few.
The newest CDL products have less visible scene lines because of the abundance of available imagery and the new processing techniques from the new See5.0 method. As a final step in the CDL development, spatial smoothing to reduce the effect of "salt and pepper" noise inherent to pixel based classification is applied through the use of a minimum mapping unit (MMU). The premise is to remove single or small groups of isolated pixels that are likely misclassified (e.g. a few scattered corn pixels within a field dominated by soybean pixels). Within cropland areas, pixel groups of 10 acres (13 AWiFS pixels) or less are eliminated and replaced with the neighboring majority value. Within, non-cropland areas (which tend to be more geographically detailed) a smaller value of 4 acres (5 pixels) is used. NASS has repeatedly tested the performance of the MMU and found they typically improve map accuracies by a couple of percentage points in addition to providing a more visually pleasing output. Furthermore, the 13 and 5 values represent optimal parameters appropriate for use throughout the central US growing region. |