CONCOR is a package which can be used to identify and correct invalid and
inconsistent data being prepared for tabulation and analysis. It is a
tool for editing population and housing data from national censuses as well
as from a variety of other statistical activities. CONCOR can be used as
a stand-alone package or interactively with CENTRY to edit data during
data entry.
CONCOR is the editing component of the Integrated Microcomputer Processing
System (IMPS), a series of software packages for entry, editing,
tabulation, estimation, analysis, and dissemination of census and survey
data. IMPS was developed by the U.S. Bureau of the Census. Distribution
of IMPS is funded jointly by the Bureau of the Census and the U.S. Agency
for International Development (USAID).
Editing Capabilities
- Structure Checks - These checks ascertain that all records which
should be present for a particular questionnaire are present, in the
correct order, and no extra records are included.
- Valid Value Checks - These checks are designed to determine whether
a response has a value which is within the valid limits for that response.
These checks are concerned with single responses.
- Consistency Checks - In these checks, two or more responses in a
questionnaire are compared for their consistency. The responses may be in
the same record or in different records within a questionnaire.
- Automatic Correction - When a census or other large survey is being
processed, it would be unduly cumbersome to make most data corrections
by visually examining the errors. CONCOR provides the facility to find
incorrect or inconsistent data and make corrections according to your
specifications. The "hot deck" method of correction is one of several
methods available.
- Corrected File - CONCOR can create a file of corrected data which is
identical in format to the unedited file. In fact, CONCOR can reedit the
edited file to ensure that no inconsistencies were introduced in the
original CONCOR run.
- Lookup File - CONCOR can match information on the primary input
file with information on a secondary input file or "lookup" file. This
allows you to look up valid codes, sample weights, or other information
which is required by the edits or needs to be added to the data file
before tabulation.
- Extract File - CONCOR can create a file which contains any
combination of data from the input file, lookup file, or derived variables.
This allows you to create a reformatted data file or a specialized
report.
- Edit Reports - CONCOR generates comprehensive statistics about the
edit tests performed and the number of imputations made. These
statistical reports are generated at your discretion.
Edit Reports
CONCOR can produce three types of reports:
- Summary - This report gives the number and percent of errors found
or corrections made by CONCOR. It presents the basic information on what
happened during the edit process and is presented in the sequence in which
the edits are coded within the CONCOR program.
- Questionnaire - This report lists some or all of the data items
from a record or questionnaire. It is useful in testing edits and can be
used to list questionnaires or records which require manual correction.
- Frequency of Imputation - This report gives the number and percent
of the values used as corrections for particular data items. It allows a
more detailed look at the corrections made.
The generation of each of these reports is under your complete control.
The formatting of the reports is done by CONCOR. Reports may be repeated
by geographic area or produced once for the entire data file.
Input Data Requirements
- The data file must be sequential.
- All items must be in fixed positions on the records.
- All records for a particular questionnaire must be physically adjacent on the file.
- A unique questionnaire identification code must be present in the same
location on each record. All records belonging to the same questionnaire
must have the same code. This condition may be satisfied by a collection
of fields giving geographic coding information and a household number
which is unique within the lowest geographic level.
- If there is more than one record type, each record must have a record
type code in a fixed position and of a fixed length.
- If you wish to have CONCOR produce edit statistics separately for
unique geographic areas, the questionnaires must be grouped by
geographic unit.
Processing Methodology
CONCOR provides a structured programming language for writing edits. A
comprehensive analysis of your program is performed to clearly identify
and explain syntax errors before the program is used to edit the data
file. CONCOR programs can be easily maintained and can be understood
by both programmers and subject-matter specialists.
Through the CONCOR language, you describe the data file to be edited, the
types of edit statistics to be collected and printed, and the edits to
be performed. CONCOR analyzes your instructions for syntactic correctness.
If all statements are correct, it produces a COBOL source program. This
program is compiled and changed into an executable format. It can then
be tested against sample data to ensure that it is logically correct, and
it produces the desired results.
The process of testing, correcting, and retesting can be repeated until a
logically correct edit program is produced. This final version of the
edit program can then used to edit the data from a census or survey. The
edit program reads a "raw" data file, modifies it according to your
edit instructions, and produces a "clean" data file. The "clean" data
file does not overwrite the "raw" data file. Moreover, it is not necessary
to modify the raw file and create a clean file if CONCOR is only being used
to identify errors. The edit program also generates the requested edit
statistics.
Data Dictionary
To use CONCOR, a description of the file to be processed must be given. To
create this description, you use the Data Dictionary component of IMPS,
which is provided with CONCOR. The Data Dictionary lets you give a name to
each data item in the file, specify its location within a data record and
define the values it may have. These values are used by CONCOR to test
items for valid values.
Performance
The speed of CONCOR depends on the number of edits being performed, the
organization of the file being edited, and the type of microcomputer being
used. On an 80386 microcomputer running at 20 MHz, CONCOR runs at a speed
of about 30,000 records per minute, editing a census data file containing
about 30 questions using typical edits.
Hardware and Software
- IBM PC or compatible
- 10M bytes of disk storage
- Printer capable of 132 characters per line
- alia COBOL 2.0 or higher
The Realia COBOL compiler is NOT supplied with CONCOR. It must be
purchased from a software supplier. The cost is approximately $1000 (US).
For more information about Realia COBOL contact:
| |
Computer Associates Realia
2 Executive Drive
Ft. Lee, New Jersey 07024, USA
Telephone:1 (201) 592-0009
Fax: 1 (201) 585-6746 |
