By Marianne W. Zawitz, Chief, Publications and Electronic Dissemination
August, 2004
Contents: Background | Survey strategy | Findings | Conclusions | Methodology
Tom Hester, Chief, Publication Development and Verification; Priscilla Middleton, Information Specialist; and the NCJRS staff made significant contributions to these surveys and the report.
The Bureau of Justice Statistics (BJS), the official source of crime and justice statistics in the United States, disseminates the data it collects to Federal, State, and local policy makers, criminal justice practitioners, the media, educators, researchers, students, and the general public. BJS conducts surveys on criminal victimization and the operation of the U.S. justice system, which consists of over 50,000 different agencies.
BJS periodically surveys its regular customers; surveys were conducted in 1994, 1997 and 2003. With the advent of the Internet, the earlier surveys focused on the interest and ability of customers to move to electronic dissemination methods. Now that access to the Internet and familiarity with associated technologies are no longer issues for most BJS customers, we have turned to finding out what our customers do, whether they use statistics to perform their functions, and what information resources they use.
BJS publishes all materials in electronic formats; the release is on the Internet and, if warranted, publications are subsequently printed. Since the website was launched in 1996, usage has increased steadily. The BJS website hosts as many as 18,000 visitors a day. Over a third of our visitors are referred by search engines. On the other hand, over 20% are not referred by other sites but get to our site by bookmarks or direct entry of the URL. Around 18% of our website visitors are repeat visitors. We know the Internet has expanded our audience, and key audiences also access our information online.
As our website has matured, our paper distribution has declined, dwarfed by web activity. BJS continues to distribute a fairly large number of short paper publications to specific groups of customers through bulk mailings, a low cost method of paper distribution. Our distribution strategy has been to concentrate this bulk distribution to those customers who have registered with the National Criminal Justice Reference Service (NCJRS), a clearinghouse service sponsored by a number of Federal agencies with an interest in crime and justice.
Approximately 40,000 customers were registered when the study was conducted. Almost half of the registered users work in a justice agency. Less than 1% were affiliated with the media.
Organizational affiliation |
Percent of registered users |
---|---|
Criminal justice agency | 45.1% |
Educational association/university | 16.0% |
Community organization | 13.1% |
Social services | 4.6% |
Private sector | 4.0% |
Health services | 2.5% |
Research/statistics organization | 2.5% |
Executive branch | 1.4% |
Professional association/society | 1.1% |
Media organization | 0.8% |
Legislative branch | 0.6% |
Indian tribal community | 0.5% |
None | 7.8% |
100.0% |
Most of the registered users were either administrators/managers or staff. At the time of the surveys, the registered user list had the following composition:
Position | Percent of registered users |
---|---|
Administrator/ manager | 41.1% |
Staff | 29.1% |
Educator | 10.0% |
Researcher | 7.2% |
Student | 6.2% |
Elected/appointed official | 2.8% |
Volunteer | 1.6% |
General public | 0.5% |
Other and no response | 1.4% |
Total | 100.0% |
Registered users select topics of interest (law enforcement, violence, etc.) as well as type of information they are interested in (statistics, research, etc.) Forty percent of registered users indicated an interest in statistics. Customers who register are told they will automatically be sent documents from time to time, and they are subscribed to an E-mail newsletter announcing new publications.
In addition to the registered users, NCJRS maintains lists of targeted mail recipients, such as elected and appointed officials, association memberships, and related rosters. BJS has used these lists in their dissemination program but to a lesser degree than the registered user list focusing on Federal officials including the Congress, agency officials, and professional associations.
Many people on the registered and targeted lists are both users of our data and respondents to our surveys.
Even though the number of web users dwarfs our paper distributions, we decided not to survey web users since such surveys suffer from extremely low response rates and other problems which affect the ability to infer results. We decided to survey paper recipients since we have much more information about them and know that they represent key audiences. In addition, we were able to conduct a sample survey of registered users so we could empirically assess quality of the results.
Past surveys had focused on registered users but not on user behavior with relation to specific reports from BJS. Therefore, we decided to tie the current effort to specific documents, selecting recent documents in topical areas to ensure coverage of audiences for subject matter such as law enforcement, courts, corrections and victimization. These areas of interest correspond to the primary audiences for BJS reports.
We assumed that members of each group were very different in terms of what they did and how they retrieved information. The following documents were selected for the survey:
Registration selections | ||||
---|---|---|---|---|
Title | Length in pages |
Topical interests | Statistics interest |
|
Local Police Departments 2000 | 33 | Law enforcement | ||
Intimate Partner Violence, 1993-2001 | 2 | Victims, Family Violence | ||
Prison and Jail Inmates
|
12 | Corrections | X | |
Juvenile Felony Defendants
in Criminal Courts,1998 |
12 | Courts, Juvenile Justice | X |
Since the registered user lists include phone numbers for most users, we decided to conduct a phone survey of registered users. The detailed information concerning the characteristics of the registered users also allowed us to do a sample survey of 300 registered users for each publication. For targeted users, we had only mailing addresses, so we conducted a mail survey. Two weeks after each report was sent by the bulk mail, a sample of registered users were called and a mail survey went to all of the targeted users except for "certainty units" like members of Congress. The following table summarizes the universe and responses for each report:
Registered users | Targeted users | ||||
---|---|---|---|---|---|
Reports | Sent the report | Successful interview | Sent the report | Returned survey | Total respondents |
Local Police Departments, 2000 |
17,391 | 300 | 2,800 | 352 | 652 |
Intimate Partner Violence, 1993-2001 |
19,019 | 300 | 3,591 | 389 | 689 |
Prison and Jail Inmates at Midyear 2002 |
24,355 | 300 | 3,242 | 587 | 887 |
Juvenile Felony Defendants in Criminal Courts |
8,988 | 300 | 6,010 | 258 | 558 |
Total | 69,753 | 1,200 | 15,643 | 1,586 | 2,786 |
The registered customer profile drawn from the NCJRS Customer Relationship Management System data resembles the profile of the sampled users who completed the phone survey. The only exceptions were that smaller proportions of elected officials and students responded to the phone survey than were in the original list. We anticipated these proportions would be lower due to the difficulty in contacting elected officials directly by phone and in contacting students in the summer when school is out of session. Additional detail on the methodology is available at the end of this paper.
We also fielded a survey of people who ordered these specific publications. These mail surveys were sent after the order was fulfilled. Because the number of orders was small and the response rate was low, the responses from this survey are not included here.
We analyzed the results by publication and by type of recipient. Unless otherwise noted, responses between the registered user recipients of the various reports did not differ significantly. Since the survey of the targeted mail recipients had a very low response rate, the findings presented are not representative of the entire group.
Most customers scanned the documents they received. Of the registered users, few read the document thoroughly. Over one-fourth of registered users did not receive the document or were not sure that they received it. A higher proportion of targeted mail recipients than registered users had read the document thoroughly or had scanned it. This result is probably due to the fact that the targeted mail survey was a voluntary survey so motivated recipients were more likely to respond.
Registered users |
Targeted mail recipients |
|
---|---|---|
Read it thoroughly | 6% | 25% |
Scanned it | 47% | 59% |
Did not read it | 19% | 9% |
Not sure | 18% | 2% |
Never received it | 10% | 5% |
Percent of respondents who acknowledged receipt | |||
---|---|---|---|
Registered users | Targeted mail recipients | ||
Gave it to someone else | 9% | 11% | |
Circulated it to others | 20% | 39% | |
Kept it for reference | 66% | 43% | |
Threw it away | 5% | 7% |
Percent of respondents who threw document away | ||
---|---|---|
Reason for throwing it away | Registered users | Targeted mail recipients |
Already read the online version | 3% | 0% |
Not interested in this topic | 60% | 25% |
Never use statistics | 10% | 21% |
No space for it | 10% | 21% |
Will use online version if needed again | 18% | 34% |
Percent who indicated
knowledge of the publication before receiving it |
||
---|---|---|
Registered users |
Targeted recipients |
|
Total | 36% | 23% |
Prison and Jail Inmates at Midyear | 43% | 27% |
Juvenile Felony Defendants | 37% | 16% |
Local Police Departments | 34% | 28% |
Intimate Partner Victims | 28% | 22% |
Further probing in the survey revealed that of registered users who knew about the report before receiving it, a third knew about it through email notification. Of targeted users, a fifth knew about it by email notification. This represents 9% of the all registered users who were surveyed and 5% of the targeted mail recipients who responded. Browsing the Internet was the second most frequently given method of finding out about a document for registered users. Few reported that they learned about the report by other means, which was to be expected since most other means had not been actively pursued as marketing techniques. Only Prison and Jail Inmates at Midyear 2002 was released by a news release, and there was no direct attempt to use article placement or advertising in other professional publications.
NCJRS conducts regular notification of registered users through an electronic biweekly news letter, JUSTINFO. In addition, BJS operates a ListServe which immediately notifies interested subscribers of all statistical releases from three agencies in the Department of Justice. BJS announces their publications and releases in both. Therefore, the small number who reported that they knew of the report through email notices was disappointing. It is possible that they did not read the notification or that they did not remember the announcement since the electronic announcements occur as soon after the publication is available on the Internet, several months before print copies are available.
Percent of all respondents |
||||
---|---|---|---|---|
How they knew it was available | Registered users |
Targeted mail recipients | ||
E-mail notification | 34% | 21% | ||
Browsing the Internet | 27% | 12% | ||
Some other way | 25% | 30% | ||
From professional publication | 6% | 21% | ||
From a colleague | 5% | 10% | ||
News report | 3% | 6% | ||
Percent who would have ordered the document | ||
---|---|---|
Registered users |
Targeted mail recipients | |
Total | 54% | 31% |
Local Police Departments | 52% | 37% |
Intimate Partner Victims | 54% | 36% |
Prison and Jail Inmates at Midyear | 49% | 26% |
Juvenile Felony Defendants | 60% | 24% |
The results for the registered users show that they represent a defined market for these documents. The results from the targeted mail respondents present a less clear picture since they were more likely to read the document thoroughly and were more likely to share it.
These surveys were conducted during a time when citizens could order most documents from NCJRS at no cost. Since that time, limits have been placed on the number of documents that can be ordered for free and on which documents are available for free. Therefore, the willingness of the respondents to order documents may change when faced with fees.
Feelings on being sent unsolicited materials: |
Registered users |
Targeted mail recipients |
|
---|---|---|---|
Very positive | 61% | 56% | |
Somewhat positive | 30% | 26% | |
Neutral | 7% | 15% | |
Somewhat negative | 2% | 2% | |
Very negative | 1% | 1% | |
100% | 100% |
Activities | Registered Users | Targeted Mail Recipients | ||
---|---|---|---|---|
Writes Reports | 83.9 | % | 53.0 | % |
Gives presentations | 82.8 | 57.1 | ||
Plans | 78.1 | 54.8 | ||
Trains | 76.3 | 46.8 | ||
Researches | 67.7 | 43.9 | ||
Prepares programs | 66.3 | 52.0 | ||
Conducts Conferences | 61.9 | 25.8 | ||
Teaches | 60.4 | 43.9 | ||
Writes proposals | 52.8 | 40.5 | ||
Drafts legislation | 32.2 | 19.9 | ||
Media | 29.8 | 18.3 | ||
Does school work | 25.1 | 6.8 | ||
Other activities | 2.8 | 19.5 |
Would use the report they received to perform their frequent activities | ||||
---|---|---|---|---|
Report | Registered users |
Targeted |
||
Total | 72% | 86% | ||
Local Police Departments | 80% | 89% | ||
Intimate Partner Victims | 75% | 81% | ||
Prison and Jail Inmates at Midyear | 63% | 89% | ||
Juvenile Felony Defendants | 69% | 85% |
Seeks information from: | Registered users |
Targeted mail recipients | ||
---|---|---|---|---|
Looks for information on the Internet | 94.7 | % | 77.6 | % |
Consults other materials | 94.0 | 82.3 | ||
Asks a colleague for information | 80.0 | 44.5 | ||
Contacts NCJRS for information | 76.8 | 40.4 | ||
Directly contacts source | 72.2 | 52.5 | ||
Goes to the library for information | 43.1 | 21.2 | ||
Note: Respondents could select more than one place. |
We were also interested in what type of information our customers usually use. For registered users, 95% used the Internet followed by other government sources (92%) and other BJS reports (90%). In general, registered users were more likely than targeted mail recipients to use each type of information source. The most frequently cited sources for targeted mail recipients were journals and the Internet. The least likely source for both groups was magazines, although almost two-thirds of the registered users listed that as a possible source.
Percent who use source | ||||
---|---|---|---|---|
Information sources | Registered users |
Targeted mail | ||
The Internet | 95.1 | % | 75.3 | % |
Other government sources | 91.7 | 68.3 | ||
Other BJS reports | 90.3 | 64.3 | ||
Journals | 86.5 | 76.4 | ||
Newspapers | 73.1 | 44.9 | ||
Own research | 71.4 | 56.7 | ||
Magazines | 64.0 | 42.7 | ||
Note: Respondents could select more than one source. |
In general, the results of the surveys were very encouraging to BJS. The customers we are directly reaching use statistics and use BJS reports. The current approach of sending bulk mailings to registered users with additional augmentation from targeted lists appears to be successful. That most recipients keep or share the documents confirms the efficacy of this approach. Because there were few differences between the registered user audiences of distinct BJS products, we conclude that we are serving those audiences about equally well and that we have few obvious gaps to fill.
Email notification appears to fall below our expectations. We need more research on the efficacy of using other types of notification including the mailing of documents like catalogs and postcards.
This survey does not tell us if we are reaching most of our intended audiences. We need to understand coverage of target audiences in registered users. We may want to convert appropriate candidates among the targeted users into registered users. In addition, we need to find other ways to assess our outreach to those audiences that are not covered well by NCJRS such as the media, secondary school students and teachers, and the general public.
Clearly the findings have implications for how our statistics are packaged. Customer attitudes did not differ by publication even though the length of the publications and topics were diverse. Most users engage in activities that involve locating a few numbers rather knowing everything about a subject. A recent study supported by BJS showed that simply transferring publications to the web fails because users approach content on the web differently than content on paper.[1] We need further research on the best methods of delivering our content.
The surveys show that the Internet is most likely place to seek information. Therefore, we need to continue to support customers in their search to discover answers online. Traditionally, we have relied on intermediaries such a librarians and the NCJRS information center to convey cautions on proper use of statistics. The use of unmediated sources like the Internet has implications for proper use and understanding of statistics. Our customers also rely on other sources such as journals and, to a lesser extent, newspapers. We need to assess how these outlets can be leveraged to alert our customers to available information.
Traditionally, we focused considerable effort into getting our materials in libraries. The customers who responded to our surveys were less likely to use libraries when seeking information. We need to examine our dissemination to libraries and determine which audiences are best served through libraries.
The survey was conducted during the summer of 2003. The NCJRS staff conducted the phone surveys about two weeks after each publication was sent by mail. Registered users with phone numbers in the NCJRS database were randomly contacted. The lists for each publication were cross checked so that no registered user could be contacted more than once. NCJRS staff members were instructed to select another user if the phone number was not working, if they encountered a busy signal or an answering machine, or if the user was not available. The results of the phone survey were as follows:
Sent report | Called | Calls that did not get through |
Refused | Incomplete Survey |
Completed Survey |
||
---|---|---|---|---|---|---|---|
Local Police Departments 2000 |
17,391 | 2,962 | 2,656 | 60 | 5 | 300 | |
Inmate Partner Violence 1993-2001 |
19,019 | 2,407 | 2,075 | 66 | 5 | 300 | |
Prison and Jail Inmates at Midyear 2002 |
24,355 | 2,056 | 1,760 | 73 | 3 | 300 | |
Juvenile Felony Defendants in Criminal Courts |
8,988 | 2,424 | 2,050 | 75 | 4 | 300 |
A mail survey was sent to targeted mail users about two weeks after the mailing. Some targeted mail users were excluded as distribution was to these groups was not considered optional. The excluded groups were Congress, Sourcebook contributors, Statistical Analysis Centers, State Uniform Crime Reports Programs, BJS VIPs (interested persons who request all publications), and certain Federal Libraries. The targeted mailing response is as follows:
Number targeted mail recipients | Number of responses | Targeted response rate | |
---|---|---|---|
Local Police Departments 2000 | 2,800 | 352 | 13% |
Intimate Partner Victims 1993-2001 | 3,591 | 389 | 11% |
Prison and Jail Inmates at Midyear 2002 | 3,242 | 587 | 18% |
Juvenile Felony Defendants in Criminal Courts | 6,010 | 258 | 4% |
[1] Carol A. Hert, Ph.D., "Positioning Sourcebook of Criminal Justice Statistics for the 21st Century," presentation to BJS, July 8, 2004.