Tom Hester, Chief, Publication Development and Verification; Priscilla Middleton, Information Specialist; and the NCJRS staff made significant contributions to these surveys and the report.

The Bureau of Justice Statistics (BJS), the official source of crime and justice statistics in the United States, disseminates the data it collects to Federal, State, and local policy makers, criminal justice practitioners, the media, educators, researchers, students, and the general public. BJS conducts surveys on criminal victimization and the operation of the U.S. justice system, which consists of over 50,000 different agencies.

BJS periodically surveys its regular customers; surveys were conducted in 1994, 1997 and 2003. With the advent of the Internet, the earlier surveys focused on the interest and ability of customers to move to electronic dissemination methods. Now that access to the Internet and familiarity with associated technologies are no longer issues for most BJS customers, we have turned to finding out what our customers do, whether they use statistics to perform their functions, and what information resources they use.

BJS publishes all materials in electronic formats; the release is on the Internet and, if warranted, publications are subsequently printed. Since the website was launched in 1996, usage has increased steadily. The BJS website hosts as many as 18,000 visitors a day. Over a third of our visitors are referred by search engines. On the other hand, over 20% are not referred by other sites but get to our site by bookmarks or direct entry of the URL. Around 18% of our website visitors are repeat visitors. We know the Internet has expanded our audience, and key audiences also access our information online.

As our website has matured, our paper distribution has declined, dwarfed by web activity. BJS continues to distribute a fairly large number of short paper publications to specific groups of customers through bulk mailings, a low cost method of paper distribution. Our distribution strategy has been to concentrate this bulk distribution to those customers who have registered with the National Criminal Justice Reference Service (NCJRS), a clearinghouse service sponsored by a number of Federal agencies with an interest in crime and justice.

Approximately 40,000 customers were registered when the study was conducted. Almost half of the registered users work in a justice agency. Less than 1% were affiliated with the media.

Organizational affiliation	Percent of registered users

Criminal justice agency	45.1%
Educational association/university	16.0%
Community organization	13.1%
Social services	4.6%
Private sector	4.0%
Health services	2.5%
Research/statistics organization	2.5%
Executive branch	1.4%
Professional association/society	1.1%
Media organization	0.8%
Legislative branch	0.6%
Indian tribal community	0.5%
None	7.8%
	100.0%

Most of the registered users were either administrators/managers or staff. At the time of the surveys, the registered user list had the following composition:

Position	Percent of registered users

Administrator/ manager	41.1%
Staff	29.1%
Educator	10.0%
Researcher	7.2%
Student	6.2%
Elected/appointed official	2.8%
Volunteer	1.6%
General public	0.5%
Other and no response	1.4%
Total	100.0%

Registered users select topics of interest (law enforcement, violence, etc.) as well as type of information they are interested in (statistics, research, etc.) Forty percent of registered users indicated an interest in statistics. Customers who register are told they will automatically be sent documents from time to time, and they are subscribed to an E-mail newsletter announcing new publications.

In addition to the registered users, NCJRS maintains lists of targeted mail recipients, such as elected and appointed officials, association memberships, and related rosters. BJS has used these lists in their dissemination program but to a lesser degree than the registered user list focusing on Federal officials including the Congress, agency officials, and professional associations.

Many people on the registered and targeted lists are both users of our data and respondents to our surveys.

To the top

Survey strategy

Even though the number of web users dwarfs our paper distributions, we decided not to survey web users since such surveys suffer from extremely low response rates and other problems which affect the ability to infer results. We decided to survey paper recipients since we have much more information about them and know that they represent key audiences. In addition, we were able to conduct a sample survey of registered users so we could empirically assess quality of the results.

Past surveys had focused on registered users but not on user behavior with relation to specific reports from BJS. Therefore, we decided to tie the current effort to specific documents, selecting recent documents in topical areas to ensure coverage of audiences for subject matter such as law enforcement, courts, corrections and victimization. These areas of interest correspond to the primary audiences for BJS reports.

We assumed that members of each group were very different in terms of what they did and how they retrieved information. The following documents were selected for the survey:

Title	Length in pages	Topical interests	Statistics interest
		Registration selections


Local Police Departments 2000	33	Law enforcement
Intimate Partner Violence, 1993-2001	2	Victims, Family Violence
Prison and Jail Inmates at Midyear 2002	12	Corrections	X
Juvenile Felony Defendants in Criminal Courts,1998	12	Courts, Juvenile Justice	X

Since the registered user lists include phone numbers for most users, we decided to conduct a phone survey of registered users. The detailed information concerning the characteristics of the registered users also allowed us to do a sample survey of 300 registered users for each publication. For targeted users, we had only mailing addresses, so we conducted a mail survey. Two weeks after each report was sent by the bulk mail, a sample of registered users were called and a mail survey went to all of the targeted users except for "certainty units" like members of Congress. The following table summarizes the universe and responses for each report:

Reports	Sent the report	Successful interview	Sent the report	Returned survey	Total respondents
	Registered users		Targeted users


Local Police Departments, 2000	17,391	300	2,800	352	652
Intimate Partner Violence, 1993-2001	19,019	300	3,591	389	689
Prison and Jail Inmates at Midyear 2002	24,355	300	3,242	587	887
Juvenile Felony Defendants in Criminal Courts	8,988	300	6,010	258	558
Total	69,753	1,200	15,643	1,586	2,786

The registered customer profile drawn from the NCJRS Customer Relationship Management System data resembles the profile of the sampled users who completed the phone survey. The only exceptions were that smaller proportions of elected officials and students responded to the phone survey than were in the original list. We anticipated these proportions would be lower due to the difficulty in contacting elected officials directly by phone and in contacting students in the summer when school is out of session. Additional detail on the methodology is available at the end of this paper.

We also fielded a survey of people who ordered these specific publications. These mail surveys were sent after the order was fulfilled. Because the number of orders was small and the response rate was low, the responses from this survey are not included here.

We analyzed the results by publication and by type of recipient. Unless otherwise noted, responses between the registered user recipients of the various reports did not differ significantly. Since the survey of the targeted mail recipients had a very low response rate, the findings presented are not representative of the entire group.

To the top

Survey findings

What did customers do with the documents they received?

Most customers scanned the documents they received. Of the registered users, few read the document thoroughly. Over one-fourth of registered users did not receive the document or were not sure that they received it. A higher proportion of targeted mail recipients than registered users had read the document thoroughly or had scanned it. This result is probably due to the fact that the targeted mail survey was a voluntary survey so motivated recipients were more likely to respond.

	Registered users	Targeted mail recipients

Read it thoroughly	6%	25%
Scanned it	47%	59%
Did not read it	19%	9%
Not sure	18%	2%
Never received it	10%	5%

After their initial review, what did they do with the report?

Until this survey, BJS had little information on whether the documents they sent to customers were being used or thrown away. Clearly, if users were throwing our documents away, the continued practice of sending unsolicited materials would need to be reconsidered. The survey showed that most respondents kept the document for reference and few threw it away. All of these documents contained statistical content which may be why they were kept for reference. Materials intended to notify users of publications, events, and other opportunities that do not contain reference content may not be treated the same way.

We were also interested in the secondary distribution of our materials to others who were not the initial recipient. Of the registered users, almost a third gave the documents to others or circulated them, while half of the targeted users shared the documents. In either case, sharing the documents expands the number of people who get direct access and increases the outreach from a single mailing.


	Percent of respondents who acknowledged receipt
	Registered users	Targeted mail recipients

Gave it to someone else	9%	11%
Circulated it to others	20%	39%
Kept it for reference	66%	43%
Threw it away	5%	7%

If they threw it away, why?

Of registered users who threw the document away, 60% said they were not interested in the topic. Some indicated that they threw it away because they could use the online version if needed later. Few said they never use statistics or had no space for the document. The majority of the registered users are selected to receive a document by their expressed topical preferences and an interest in statistics. Perhaps their lack of interest in the topic reflected a mismatch; the topics used in registration are very broad and the documents represented a narrower subtopic.

Reason for throwing it away	Registered users	Targeted mail recipients
	Percent of respondents who threw document away


Already read the online version	3%	0%
Not interested in this topic	60%	25%
Never use statistics	10%	21%
No space for it	10%	21%
Will use online version if needed again	18%	34%

Did they know about it before they received it?

Slightly over a third of the registered users who received the document reported knowing about the document before they received it. Among targeted users, less than a quarter knew about it in advance of receipt. Prison and Jail Inmates at Midyear had the highest percentage of registered users with prior knowledge. This is probably because this is an annual publication they may have received before. In addition, some may be contributors to the surveys the report is based on.

	Percent who indicated knowledge of the publication before receiving it

	Registered users	Targeted mail recipients

Total	36%	23%
Prison and Jail Inmates at Midyear	43%	27%
Juvenile Felony Defendants	37%	16%
Local Police Departments	34%	28%
Intimate Partner Victims	28%	22%

Further probing in the survey revealed that of registered users who knew about the report before receiving it, a third knew about it through email notification. Of targeted users, a fifth knew about it by email notification. This represents 9% of the all registered users who were surveyed and 5% of the targeted mail recipients who responded. Browsing the Internet was the second most frequently given method of finding out about a document for registered users. Few reported that they learned about the report by other means, which was to be expected since most other means had not been actively pursued as marketing techniques. Only Prison and Jail Inmates at Midyear 2002 was released by a news release, and there was no direct attempt to use article placement or advertising in other professional publications.

NCJRS conducts regular notification of registered users through an electronic biweekly news letter, JUSTINFO. In addition, BJS operates a ListServe which immediately notifies interested subscribers of all statistical releases from three agencies in the Department of Justice. BJS announces their publications and releases in both. Therefore, the small number who reported that they knew of the report through email notices was disappointing. It is possible that they did not read the notification or that they did not remember the announcement since the electronic announcements occur as soon after the publication is available on the Internet, several months before print copies are available.

How they knew it was available	Registered users
	Percent of all respondents


E-mail notification	34%	21%
Browsing the Internet	27%	12%
Some other way	25%	30%
From professional publication	6%	21%
From a colleague	5%	10%
News report	3%	6%

Would they have ordered this document if they had not received it?

Over half of the registered users who received the document indicated that the document they were sent was something that they would have ordered, if they had not received it. For targeted mail respondents, less than one third would have ordered it. Registered users who received Juvenile Felony Defendants were most likely to indicate that they would order the document.

	Registered users	Targeted mail recipients
	Percent who would have ordered the document


Total	54%	31%
Local Police Departments	52%	37%
Intimate Partner Victims	54%	36%
Prison and Jail Inmates at Midyear	49%	26%
Juvenile Felony Defendants	60%	24%

The results for the registered users show that they represent a defined market for these documents. The results from the targeted mail respondents present a less clear picture since they were more likely to read the document thoroughly and were more likely to share it.

These surveys were conducted during a time when citizens could order most documents from NCJRS at no cost. Since that time, limits have been placed on the number of documents that can be ordered for free and on which documents are available for free. Therefore, the willingness of the respondents to order documents may change when faced with fees.

Do they mind receiving unsolicited mail from us?

Both groups had positive feelings about being sent unsolicited materials. Few were negative about this practice. The support voiced by targeted mail recipients may be skewed due to the voluntary nature of the response. However, the registered users' positive answers show us that we can continue this approach without concern that we are intruding on our customers.

	Registered users	Targeted mail recipients

Very positive	61%	56%
Somewhat positive	30%	26%
Neutral	7%	15%
Somewhat negative	2%	2%
Very negative	1%	1%
	100%	100%

What do our customers do and do they use statistics?

What activities do they perform?

A major focus of the survey was to ask what activities customers perform. We know they are employed in criminal justice agencies, educational institutions, and other government and private organizations. However, we were uncertain about the kinds of tasks they undertake and how they use statistics. Registered users most frequently responded that they write reports, give presentations, and plan. We have anecdotal evidence that they are often involved in writing proposals; however, slightly over half said they engage in this activity, far fewer than other activities listed.

Fewer targeted users indicated performing most activities. Slightly more than half of the respondents indicated that they give presentations, plan, write reports, and prepare programs. We suspect that the differences between registered users' and targeted mail recipients' involvement in the activities is due to the different methods of data collection, phone survey vs. paper survey.

Activities	Registered Users		Targeted Mail Recipients

Writes Reports	83.9	%	53.0	%
Gives presentations	82.8		57.1
Plans	78.1		54.8
Trains	76.3		46.8
Researches	67.7		43.9
Prepares programs	66.3		52.0
Conducts Conferences	61.9		25.8
Teaches	60.4		43.9
Writes proposals	52.8		40.5
Drafts legislation	32.2		19.9
Media	29.8		18.3
Does school work	25.1		6.8
Other activities	2.8		19.5

The small number of other activities listed is an indication that the list presented was fairly exhaustive, particularly for the registered users.

Do they use statistics in performing these activities?

Ninety-two percent of the registered users indicated that they used statistics to complete the activities reported. Of targeted mail recipients, 83% said they used statistics. Since many registered users who received our documents had indicated an interest in statistics at registration, a high level of interest was expected. This high level of use by both groups suggests that our customers value statistics.

We also asked them if they would use the report that we sent them in the activities reported. Overall, 72% of registered users and 86% of the targeted mail recipients said that they would use the report we sent them. Registered users who had received Local Police Departments and Intimate Partner Victims were slightly more likely than the recipients of Prison and Jail Inmates at Midyear and Juvenile Felony Defendants to say that they would use the documents. Targeted mail recipients were more likely than registered users to say they would use each of the documents, but this may have been a function of self-selection in responding to the mail survey.

	Would use the report they received to perform their frequent activities

Report	Registered users	Targeted mail

Total	72%	86%
Local Police Departments	80%	89%
Intimate Partner Victims	75%	81%
Prison and Jail Inmates at Midyear	63%	89%
Juvenile Felony Defendants	69%	85%

Where do they seek the information they need?

As with the other multiple response categories, registered users were more likely to respond to this question. Almost 95% of the registered users said that they looked for information on the Internet, the most frequent behavior listed. Almost as many registered users would consult other materials, while 80% would ask a colleague for information. For targeted mail users, 82% reported that they would consult other materials and 78% would look for information on the Internet. Over three-quarters of the registered users would contact NCJRS and slightly fewer would directly contact the source. For targeted mail users more than half would directly contact the source of the information. They were much less likely to contact NCJRS than registered users (40% compared to 77%). For both groups, the least likely option for seeking information was going to the library.

Seeks information from:	Registered users		Targeted mail recipients

Looks for information on the Internet	94.7	%	77.6	%
Consults other materials	94.0		82.3
Asks a colleague for information	80.0		44.5
Contacts NCJRS for information	76.8		40.4
Directly contacts source	72.2		52.5
Goes to the library for information	43.1		21.2
Note: Respondents could select more than one place.

We were also interested in what type of information our customers usually use. For registered users, 95% used the Internet followed by other government sources (92%) and other BJS reports (90%). In general, registered users were more likely than targeted mail recipients to use each type of information source. The most frequently cited sources for targeted mail recipients were journals and the Internet. The least likely source for both groups was magazines, although almost two-thirds of the registered users listed that as a possible source.

Information sources	Registered users		Targeted mail
	Percent who use source


The Internet	95.1	%	75.3	%
Other government sources	91.7		68.3
Other BJS reports	90.3		64.3
Journals	86.5		76.4
Newspapers	73.1		44.9
Own research	71.4		56.7
Magazines	64.0		42.7
Note: Respondents could select more than one source.

To the top

Conclusions

In general, the results of the surveys were very encouraging to BJS. The customers we are directly reaching use statistics and use BJS reports. The current approach of sending bulk mailings to registered users with additional augmentation from targeted lists appears to be successful. That most recipients keep or share the documents confirms the efficacy of this approach. Because there were few differences between the registered user audiences of distinct BJS products, we conclude that we are serving those audiences about equally well and that we have few obvious gaps to fill.

Email notification appears to fall below our expectations. We need more research on the efficacy of using other types of notification including the mailing of documents like catalogs and postcards.

This survey does not tell us if we are reaching most of our intended audiences. We need to understand coverage of target audiences in registered users. We may want to convert appropriate candidates among the targeted users into registered users. In addition, we need to find other ways to assess our outreach to those audiences that are not covered well by NCJRS such as the media, secondary school students and teachers, and the general public.

Clearly the findings have implications for how our statistics are packaged. Customer attitudes did not differ by publication even though the length of the publications and topics were diverse. Most users engage in activities that involve locating a few numbers rather knowing everything about a subject. A recent study supported by BJS showed that simply transferring publications to the web fails because users approach content on the web differently than content on paper.^[1] We need further research on the best methods of delivering our content.

The surveys show that the Internet is most likely place to seek information. Therefore, we need to continue to support customers in their search to discover answers online. Traditionally, we have relied on intermediaries such a librarians and the NCJRS information center to convey cautions on proper use of statistics. The use of unmediated sources like the Internet has implications for proper use and understanding of statistics. Our customers also rely on other sources such as journals and, to a lesser extent, newspapers. We need to assess how these outlets can be leveraged to alert our customers to available information.

Traditionally, we focused considerable effort into getting our materials in libraries. The customers who responded to our surveys were less likely to use libraries when seeking information. We need to examine our dissemination to libraries and determine which audiences are best served through libraries.

To the top

Methodology

The survey was conducted during the summer of 2003. The NCJRS staff conducted the phone surveys about two weeks after each publication was sent by mail. Registered users with phone numbers in the NCJRS database were randomly contacted. The lists for each publication were cross checked so that no registered user could be contacted more than once. NCJRS staff members were instructed to select another user if the phone number was not working, if they encountered a busy signal or an answering machine, or if the user was not available. The results of the phone survey were as follows:

	Sent report	Called	Calls that did not get through	Refused	Incomplete Survey	Completed Survey

Local Police Departments 2000	17,391	2,962	2,656	60	5	300
Inmate Partner Violence 1993-2001	19,019	2,407	2,075	66	5	300
Prison and Jail Inmates at Midyear 2002	24,355	2,056	1,760	73	3	300
Juvenile Felony Defendants in Criminal Courts	8,988	2,424	2,050	75	4	300

A mail survey was sent to targeted mail users about two weeks after the mailing. Some targeted mail users were excluded as distribution was to these groups was not considered optional. The excluded groups were Congress, Sourcebook contributors, Statistical Analysis Centers, State Uniform Crime Reports Programs, BJS VIPs (interested persons who request all publications), and certain Federal Libraries. The targeted mailing response is as follows:

	Number targeted mail recipients	Number of responses	Targeted response rate

Local Police Departments 2000	2,800	352	13%
Intimate Partner Victims 1993-2001	3,591	389	11%
Prison and Jail Inmates at Midyear 2002	3,242	587	18%
Juvenile Felony Defendants in Criminal Courts	6,010	258	4%

^[1] Carol A. Hert, Ph.D., "Positioning Sourcebook of Criminal Justice Statistics for the 21st Century," presentation to BJS, July 8, 2004.

Bureau of Justice Statistics Customer Feedback Survey

Survey strategy

Survey findings

What did customers do with the documents they received?

What do our customers do and do they use statistics?

Conclusions

Methodology

Bureau of Justice Statistics

Web Site

Partners

Stay Connected