BJS: Bureau of Justice Statistics

  Advanced
Search
Bureau of Justice Statistics (BJS)
Home  |  Data Quality Guidelines
Data Quality Guidelines

BJS Data Quality Guidelines

VII. Overview

BJS has published these Data Quality Guidelines to implement Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (PL 106-554). OMB has issued guidelines under Section 515 which direct the Federal agencies subject to the Paperwork Reduction Act (44 U.S.C. Chapter 35) to establish and implement written procedures to ensure and maximize the quality, utility, objectivity and integrity of the information that they disseminate. The BJS guidelines apply to a wide variety of substantive information and dissemination activities in order to meet basic information quality standards set forth by Section 515. Another purpose of these guidelines is to provide a framework under which BJS will provide persons an opportunity to seek and obtain correction of information maintained and disseminated by BJS that does not comply with these guidelines. These guidelines may be amended by the BJS Director if the BJS Director determines that good cause exists for an amendment.

A. Scope

Pursuant to the OMB guidelines (67 FR 8452) and the DOJ Information Quality Guidelines, the BJS guidelines apply to all information that is meant for public dissemination by BJS if the information is disseminated by BJS on or after October 1, 2002, regardless of when the information was first disseminated.

The BJS guidelines apply to BJS statistical products disseminated in all media (printed, electronic, or in other form). As is the intent of OMB's guidelines, the BJS guidelines focus primarily on the dissemination of substantive statistical information as enumerated in BJS' authorizing legislation rather than information pertaining to basic BJS or DOJ operations.

The BJS Data Quality Guidelines listed herein apply only to BJS statistics as disseminated directly by BJS. The guidelines do not apply to BJS statistics that are presented, manipulated, or interpreted by any entity other than BJS.

up arrowTop of this page

B. Exceptions

As stated in the above section, the BJS Data Quality Guidelines govern all justice statistics that BJS produces and disseminates for the general public, including all statistics that are featured in BJS publications, on the BJS website, and in BJS press releases. In addition to the exceptions provided in the DOJ's Information Quality Guidelines, the BJS Data Quality Guidelines do not apply to the following BJS information: information limited to government employees, grantees, or contractors; archival records, correspondence with individuals, statistical or analytic products containing a BJS disclaimer, internal BJS records, materials not intended for public distribution, hyperlinks to information disseminated by entities other than BJS, and responses to requests under the Freedom of Information Act, the Privacy Act, the Federal Advisory Committee Act, or other similar laws.

The BJS Data Quality Guidelines do not govern the entities to which the BJS website is linked. The BJS website only links to agencies and organizations with which BJS does business, including: other government agencies, professional organizations whose members provide technical advice or participate in BJS surveys, contractors, grantees, and commercial sites that provide software necessary to properly view BJS documents. It is BJS policy that links to other agencies or organizations are identified in the link title which specifies whether the site is operated by another government organization, partner sites that are funded by BJS in conducting its business, or other external sites for which BJS takes no responsibility. BJS Data Quality Guidelines only apply to the material on the BJS site and partner sites that disseminate BJS data to the extent that those sites disseminate BJS data.

BJS is committed to its mission of providing the public with justice statistics in a timely manner; however, this mission is limited by federal protections as to confidentiality of certain information and the privacy concerns of human subjects of federally funded research (discussed more fully in Section IX).

In urgent situations that may pose an imminent threat to public safety, the operations of federal, State or local criminal justice agencies, criminal justice information infrastructures, or homeland security, these requirements may be waived temporarily.

up arrowTop of this page

C. Legal Effect

The BJS Data Quality Guidelines represent the practices, priorities, and policy views of BJS. They are not applicable to any component of the DOJ other than BJS. They are not intended to be, and should not be construed as, legally binding regulations or mandates. These guidelines are intended only to inform the public of the quality controls utilized by BJS and do not create any right or benefit, substantive or procedural, enforceable at law or equity, by any party against the United States, its agencies (including the DOJ or any DOJ component), officers, or employees, or any person.

up arrowTop of this page

IX. Guidelines for Privacy and Maintaining Confidentiality of Data

Guideline 1.1 To protect the privacy and confidentiality of persons involved as subjects of BJS data collections, BJS staff will ensure that all procedures for the collection, retention, and dissemination of statistics (including the procedures of BJS grantees) are subject to federal regulations concerning data confidentiality found at 28 CFR Part 22.

Guideline 1.2 BJS must require its grantees to complete a Privacy Certificate as part of the application for BJS funding.

Guideline 1.3 All research involving human subjects will be governed by the federal protections at 28 CFR Part 46 which provide for (among other things); legally effective informed consent to be given by human subjects, approval of the research plan by an Institutional Review Board under certain circumstances, and procedures to minimize risks to the human subject.

Guideline 1.4 BJS staff will ensure that all paper and electronic data files and statistics in its possession are protected from external threats.


Discussion

As a federal statistical agency that collects and disseminates a wide array of information, BJS has taken aggressive measures to protect the privacy and confidentiality of individuals from whom they obtain information. BJS has procedures in place to ensure that information collected by BJS that is identifiable to a private person may only be used and/or revealed for the statistical or research-related purpose for which it is obtained. BJS has procedures in place to ensure that copies of such information shall not, without the consent of the person to whom the information pertains, be revealed to others who are not involved in the collection and analysis of the information.

BJS staff ensure that the identities of its human subjects are protected despite issues concerning sample size or the uniqueness of the statistical issue addressed by the data collection. Data must be aggregated as appropriate before public use datasets are disseminated. Data must not be disseminated that permits the identification of private information about an individual unless prior consent is obtained from the individual or the individual has agreed to participate in a project with knowledge that the findings cannot be expected to completely conceal subject identity.

Adequate precautions must be taken by BJS and its grantees to ensure the administrative and physical security of identifiable data. Data collection plans must be designed to preserve the anonymity of private persons to whom information relates, including, where appropriate, name-stripping, coding of data, or other similar procedures.

In order to protect information identifiable to a private person, BJS requires that a Privacy Certificate be completed pursuant to BJS regulations which state that "research or statistical information identifiable to a private person may be used only for research or statistical purposes." (28 CFR Part 22).

BJS has designated a Human Subjects Protection Officer to ensure that in accordance with the requirements in 28 CFR Part 46, BJS protects human subjects who are subject to research or the collection of data. The BJS Human Subjects Protection Officer must ensure that risks to human subjects are minimized, informed consent is obtained, and confidentiality is protected according to federal law. Pursuant to the provisions of Part 46, BJS must submit certain research projects to the review of an Institutional Review Board. The BJS Human Subjects Protection Officer is responsible for the review of each grantee's Human Subjects Screening Sheet for BJS-funded research and data collection.

Only authorized BJS staff have access to completed data collection instruments and relevant data files. Data processing of files containing identifiable data is performed on a separate secured computer server that is only accessible to BJS staff.

up arrowTop of this page

X. Guidelines for Initiating Surveys, Censuses, and other Data Collections

Guideline 2.1 BJS surveys, censuses, and other data collections may only be undertaken to address a mandate from the U.S. Congress or a U.S. Department of Justice policy initiative as determined by the U.S. Attorney General or the BJS Director.

Guideline 2.2 The BJS Director will collect and disseminate BJS statistics that address emerging information needs and highlight Department of Justice-related issues of policy relevance.

Guideline 2.3 At the discretion of the BJS Director, new data collections may be initiated and existing data collections may be modified or terminated if good cause exists.

Guideline 2.4 BJS data collections are guided by the principle that the time series associated with each data collection activity should be preserved to permit the measurement of changes over time.

Discussion

The collection of BJS statistics occurs when a determination is made by the U.S. Attorney General, the BJS Director, or the U.S. Congress that there is a policy-relevant need for a data collection. To assist the BJS Director in determining when a new data collection effort is appropriate, BJS staff will continually monitor the field of justice statistics by maintaining a dialogue with State and local partners, consulting with professional and academic experts in the field, and convening and attending conferences in order to present and discuss BJS statistics. Because the detection of changes in the justice system over time is critical to the effective evaluation of the performance of the justice system, BJS is committed to the preservation and continuation of its time series data collections.

BJS will not attempt to collect statistics pertaining to any matter that the BJS Director reasonably believes to be outside the scope of its authorizing legislation. Similarly, the BJS Director will not permit BJS staff to attempt to perform professional functions that the Director reasonably believes to be outside the scope of BJS' authorizing legislation.

up arrowTop of this page

XI. Guidelines for Survey Design and Data Collections

Guideline 3.1 Data collections will be developed and conducted with an emphasis on cost effective methodologies, proven research methods, and established statistical practice.

Guideline 3.2 All of the following elements will be determined according to established statistical standards; sample design, proper sample size, pretesting protocols for survey instruments, acceptable response rates, variables and variable definitions, burden on respondent, and data collection method (use of CAPI, personal interview, etc.).

Guideline 3.3 Data collections and survey designs will be periodically reviewed and redesigned when appropriate by BJS to ensure the collection of high quality and policy relevant statistics.

Guideline 3.4 For data collections conducted for the Department of Justice by the Bureau of the Census, the Census Bureau personnel will be governed by the Information Quality Guidelines applicable to the Census Bureau and/or the U.S. Department of Commerce.

Discussion

Prior to the collection of data (either for a new or an ongoing series), BJS staff must critically examine cost-effective solutions for both the short and long term. BJS staff should regularly analyze the statistics that are currently available in terms of relevance, quality, and timeliness to determine whether and how to collect new justice statistics. BJS staff must utilize only proven research methods and established statistical practice in every step of the BJS data collection process. In accordance with the responsibility of BJS to produce the highest quality statistics possible, the following elements of BJS statistics must be determined according to established statistical standards; sample design, proper sample size, pretesting protocols for survey instruments, acceptable response rates, variables and variable definitions, burden on respondent, and data collection method.

BJS reports and other statistical products are prepared using a variety of analytical techniques including simple tabulations of descriptive statistics, time series projections, computation of odds ratios, and utilization of life table techniques. Statistics made from sample data and cited in BJS reports are always tested to ensure that differences are not due to random variation. All analytic techniques must be reviewed by qualified BJS staff for their appropriateness to the data and the analysis. Analytic techniques that are considered to be complicated for the average reader should be clearly identified and explained in BJS reports.

Because certain Federal statistical programs need to evolve, adapt, and innovate to keep pace with the demands of the users they serve, BJS staff must remain cognizant of the need to redesign surveys if the circumstances warrant a redesign. As the 1994 redesign of the National Criminal Victimization Survey indicates, there are times when the overhaul of existing surveys may be desirable to maintain the reliability of key statistical series, especially if sources of information have changed or the way in which they are made available is modified.

For certain data collections, BJS contracts with the U.S. Census Bureau to act as the data collecting agent. When serving this function, the activities of all Census Bureau staff will be governed by the Information Quality Guidelines developed by the Census Bureau and/or the U.S. Department of Commerce.

up arrowTop of this page

XII. Guidelines for Data Transparency, Analysis, and Processing

Guideline 4.1 BJS statistical products must provide the reader with a statement of methodology that is reasonably specific as to the methods by which the statistics were generated.

Guideline 4.2 BJS statistical products must provide sufficient information to permit the reader to evaluate the strengths and weaknesses of the data.

Guideline 4.3 BJS statistical products must provide sufficient information concerning instances where data have been imputed.

Guideline 4.4 BJS staff must archive all analysis and data sets as soon as they become final in order to ensure the reproducibility of the published statistics.


Discussion

A statement of methodology must be available for all statistical products to provide a substantial measure of transparency to the statistical findings that BJS publishes. Due to the varying lengths of BJS reports, special reports, and fact sheets, resource constraints may substantially limit the statement of methodology; however, BJS reports and other statistical products must provide the reader with the ability to receive more specific information upon request. In most cases, detailed methodological descriptions of the collection will be provided in the documentation for a public use dataset.

Wherever possible, BJS publications and other statistical products should disclose the following elements: standard errors, response rates, coverage issues, modifications from previous data collections, caveats in interpreting the data, and any other information that might be valuable to the data user in interpreting BJS statistics. In order to ensure that the data are utilized in a responsible fashion by data users and policymakers, BJS must provide sufficient guidance in the interpretation of the statistics it publishes.

BJS imputation procedures must be based on theoretical and empirical considerations and must make use of the most relevant data available. Published estimates based on incomplete data for which there has been no imputation should note this fact, and should be accompanied by a statement indicating the proportion of missing data when possible. In addition, all imputed values on a data file should be clearly identified as such.

BJS staff must maintain a comprehensive archive of all survey instruments, data sets and setups and make them available to the public. Not only does this transparency enhance public confidence in BJS reports, it also provides a public record to ensure consistent treatment of variables (including weights and computations) over time for ongoing statistical series.

up arrowTop of this page

XIII. Guidelines for Content and Verification of BJS Statistics

Guideline 5.1 The objectivity of BJS statistics must be vigilantly protected at all times by BJS staff.

Guideline 5.2 All BJS reports and other statistical products must be subject to an objective and appropriate verification process conducted by qualified BJS staff other than the author of the report.

Guideline 5.3 BJS staff must be available to discuss the content of a BJS report and the methodology utilized.

Guideline 5.4 The public must have direct access to BJS staff members so that the public may obtain answers to specific questions regarding the content of BJS reports.

Guideline 5.5 Presentation of all BJS statistics must be guided by a BJS Style Guide which is available to the public.

Discussion

Because the credibility of BJS data and the integrity of BJS as a Federal statistical agency depend on the independence of BJS, the objectivity of BJS statistics must be protected at all times. Every step in the statistical process must be protected from improper external interference and the appearance thereof. Only qualified BJS staff may:

  •  conduct or oversee the collection and initial analysis of BJS data,
  •  oversee the drafting of the initial report
  •  edit the report
  •  verify the data
  •  format the report for publication
  •  oversee the publication process, and
  •  determine the release date for the report
    • .

These protections from external influence apply to all BJS-disseminated statistics and are not limited to those statistics that appear in BJS reports.

The BJS verification procedures must be conducted by a qualified BJS staff member other than the author of the report at issue. The verification procedures must include;

  •  the logic and recoding of the original analysis
  •  significance tests
  •  statistical output from the original analysis as presented in the report
  •  accuracy of text as it relates to supporting tables
  •  accuracy of all graphical depictions of data
  •  all information provided by entities outside of BJS, and
  •  matters relating to spelling, formatting, grammar, and syntax.

The content of all BJS reports and other statistical products should be capable of being explained to any interested party including the news media, researchers, interest groups, policymakers, students, and other entities inside and outside of government. In order to make the content of BJS reports comprehensible to the public, the author must be identified on the report and contact information must be provided on the BJS website, including a toll free number where more information may be obtained.

To facilitate public access to BJS staff, BJS maintains an "Ask BJS" link on the BJS website through which the public may submit questions relating to BJS statistics and reports. These questions will be routed to the qualified BJS staff member and handled in a timely manner.

In order to ensure internal consistency among all BJS publications and statistical products and facilitate their use by readers, the BJS Style Guide governs all BJS publications produced by BJS, its grantees, and contractors. The Style Guide contains publication standards and other guidance relating to use of tables, graphs, spreadsheets, punctuation, capitalization, footnotes, and bullets.

up arrowTop of this page

XIV. Guidelines for Dissemination

Guideline 6.1 BJS should ensure that all statistics it collects are disseminated to the public as widely as possible, using all appropriate media.

Guideline 6.2 BJS' dissemination policy is subject to Federal regulations concerning confidentiality and privacy (described in Section IX).

Guideline 6.3 External links featured on the BJS webpage are not governed by the BJS Data Quality Guidelines that ensure the quality, objectivity, utility, and integrity of BJS statistics.

Discussion

BJS statistics are disseminated in both paper and electronic formats. BJS makes all of its statistics easily discoverable on the BJS website where they may be downloaded for public use. There is a high degree of security for the BJS website to protect the integrity of BJS statistics from external threats.

The BJS Director must ensure that all data and reports are released to the public as soon as possible in order to comply with BJS' statutory responsibility to "provide information to the president, the Congress, the judiciary, State and local governments, and the general public on justice issues." (42 USC § 3732(c)(10)).

BJS issues press releases on about half of its publications annually. BJS maintains a list to ensure that all interested outlets are notified about new publications via email or fax. At the moment that data are released to the public, all publications and new data added to the BJS website are announced on the "What's New" web page. BJS hosts a listserve called JUSTSTATS that sends email notices to subscribers of all new releases at the time of release. BJS also contributes to JUSTINFO, a biweekly electronic newsletter hosted by NCJRS that provides notices of publications, announcements, conferences, and solicitations for certain DOJ offices.

The information posted on the BJS website includes hypertext links to information that is created and maintained by other public and private organizations in the United States and by other nations. The BJS website provides these links solely for our users' information and convenience. BJS linking policy limits links to sites of organizations that have a professional relationship with BJS including other government agencies, international statistical agencies, BJS partners, organizations that represent survey respondents, and sites that provide utilities for using BJS products.


When BJS website users select a link to an external website, they are leaving the BJS website and are subject to the data quality standards and the privacy and security policies of the owners or sponsors of the external website.


When a user follows a link to another federal agency, that agency's information quality guidelines govern the data found on the site. When a user follows a link to one of the partner sites listed on the "Related Sites" page of the BJS website, the BJS Data Quality Guidelines apply only to the official BJS data found on those sites. When a user follows a link to any other type of external website, the information contained on the external website is not under BJS control and BJS does not guarantee its accuracy, relevance, timeliness or completeness.


Although BJS exercises discretion in determining which external links to feature on its website, BJS does not necessarily endorse the organizations sponsoring linked websites and BJS does not endorse the views they express or the products and services they offer.


The vast majority of the data featured via external links are publicly available; however, in the unlikely event that an external link leads a user to copyrighted information, BJS cannot authorize the use of copyrighted materials contained in linked websites. Users must request such authorization from the sponsor of the linked website.

Previous

Next



Back to Top