Note: In order to expedite dissemination of the data, these technical notes are being presented before final copy editing. The text of the printed report may vary slightly from that shown in this unformatted WordPerfect file.

Contents

Table Page

General Notes .......................................... vii

Section A. Detailed Statistical Tables ................... 1

B. Technical Notes.................................... 85

Introduction....................................... 85

Survey Methodology................................. 87

Overview......................................... 87

Frame Creation................................... 87

Probability Proportionate to Size................ 91

Sample Allocation and Relative Standard Error

Constraints................................................ 93

Sample Selection................................. 93

The Annual Panel................................. 97

Follow-Up for Survey Nonresponse................. 97

Imputation for Item Nonresponse.................. 97

Response Rates and Mandatory Versus Voluntary

Reporting ................................................ 109

Comparability of Statistics....................... 113

Two-year Comparability.......................... 113

Historical Revisions............................ 113

Survey Definitions................................ 117

C. Survey Documents.................................. 119

The Survey of Research and Development in Industry

provides national estimates of the total expenditures on

research and development (R&D) performed within the United

States by industrial firms, whether U.S. or foreign owned.

It is a sample survey in which all R&D-performing companies,

including privately held firms, are intended to be included

or represented. All companies that are identified in the

sample year as spending more than $1 million annually on R&D

in the United States receive a survey form every year.

Information from individual companies in the sample is used

to develop national estimates on an industry-by-industry

basis. The industry R&D survey does not cover trade

associations. Although their primary mission is to serve

industry, these associations are established as nonprofit

organizations, and it is estimated that their R&D

expenditures are less than 1 percent of the industry R&D

total.

The survey is enterprise based rather than establishment

based. Therefore, all R&D data for each enterprise are

reported in the major Standard Industrial Classification

(SIC) code of the firm. Data are aggregated in this manner

for all tables in this report.

For surveys up to and including the 1991 survey, data are

based on probability samples selected about every 5 years.

In intervening years a subset of the last sample, called a

"panel," is used. The most recent probability sample prior

to the 1992 survey, was selected and first used for survey

year 1987.

The Bureau of the Census (Census), Department of Commerce,

has conducted the annual Survey of Industrial Research and

Development for the National Science Foundation (NSF) since

1957. Census staff conduct the survey under title 13 of the

U.S. Code, which prohibits publication or release of data

that may reveal information about individual companies.

Therefore, on some tables of this report, the symbol "(D)"

is used as a footnote reference to indicate that data are

being withheld to avoid possible disclosure of information

about operations of individual companies.

For detailed discussions of the history of the survey,

survey methodology, comparability of data, and survey

definitions, see section B. For more information regarding

the survey, please contact Raymond Wolfe or John Gawalt:

National Science Foundation

Division of Science Resources Studies

4201 Wilson Boulevard, Suite 965

Arlington, VA 22230

Wolfe: (703) 306-1772, ext. 6936 (voice

phone) rwolfe@nsf.gov (Internet)

Gawalt: (703) 306-1772, ext. 6940 (voice

phone)	jgawalt@nsf.gov (Internet)

NOTE

To obtain accurate historical data, use only the latest

detailed statistical tables and not data published earlier.

Data in trend tables are assembled from the most recently

completed survey cycle. Data for prior years are reviewed

for consistency with current-year responses and are revised

when necessary. For that reason, references to prior-year

data should be restricted to those published in this

document.

Classification of Reporting Units

The basic reporting unit is the company or corporate

family that includes all establishments under common

ownership or control. All research and development (R&D)

expenditures and all scientists and engineers of each

company are classified into a single standard industrial

classification (SIC) code and size category. An exception

is made for most of the companies classified in SIC code 13

(oil and gas extraction) and SIC code 50 (wholesale trade,

durable goods). Because the R&D activity of these companies

is in manufacturing product areas rather than in service

areas, their R&D expenditures are included with the SIC

classifications of the manufacturing codes that encompassed

the highest dollar value of these companies' payroll. The

R&D expenditures of all such firms in SIC 13 are included

with those of the firms in SIC 29 (petroleum). The R&D

expenditures of these types of firms in SIC 50 are allocated

to several manufacturing SIC codes.

Company Size-Class

Companies are categorized by their total number of

domestic employees. The following are the six company

size-classes used in this report:

1. Fewer than 500 employees;

2. 500 to 999 employees;

3. 1,000 to 4,999 employees;

4. 5,000 to 9,999 employees;

5. 10,000 to 24,999 employees; and

6. 25,000 or more employees.

Current/Constant Dollars

Statistics in all tables are reported in terms of current

dollars. Constant dollars are also presented in two summary

trend table, tables A-1 and A-25.

Geographic Statistics

The statistics cover only those operations in the 50

States and the District of Columbia. Company-sponsored R&D

performed outside the United States by foreign subsidiaries

of U.S. domestic companies is reported in table A-11 but is

excluded from all other tables.

Industry Classification: Census assigned an enterprise-

level SIC code to each company. A single SIC code was

assigned to multiestablishment companies based on the code

that represented the most dominant aggregated activity for

that firm in terms of total payroll. Data for the following

industry groupings are published in this report (SIC code(s)

are shown in parentheses):1

Transportation equipment

(37)

Textiles and apparel (22, 23) Motor vehicles and

motor vehicles equip. (371)

Lumber, wood products, and furniture (24, 25)

Other transportation equipment (373-75, 379)

Paper and allied products (26) Aircraft and

missiles (372, 376)3

Chemicals and allied products (28) Professional

and scientific instruments (38)

Industrial chemicals (281-82, 286)

Scientific & mechanical measuring instru. (381-82)

Drugs and medicines (283) Optical, surgical,

photo., & other instru. (383-87)

Other chemicals (284-85, 287-89)

Petroleum refining and extraction (13, 29)

Rubber products (30)

Stone, clay, and glass products (32)

Primary metals (33)

Ferrous metals and products (331-32, 3398-99)

Nonferrous metals and products (333-36)

Fabricated metal products (34)

Other manufacturing industries--printing and publishing

(27), leather products (31), and miscellaneous

manufacturing industries (39)2

Nonmanufacturing industries--mining (10-11, 14);

construction (15-17); transportation, communica-

tions, electric, gas, and sanitary services (40-42, 44-

49); wholesale and retail trade (50-51, 53-54, 56);

finance, insurance, and real estate (60, 62-63);

personal and business services (72-73); motion

pictures (78); health services (806-07); and engineer-

ing, accounting, research, management, and related

services (87)

Percentages:

Percentages were calculated on the basis of thousands

of dollars and may differ from those based on the rounded

figures shown.

Rounding:

Because of rounding, details may not add to totals

shown.

Suppression of Statistics:

Census conducts the survey under title 13 of the U.S.

Code, which prohibits publication or release of data that

may reveal information about individual companies. Also,

missing data are imputed for some data items. Therefore,

data in some of the table cells may have been deleted and

replaced with one of the following notations:

"(D)," which indicates that data are being withheld to

avoid possible disclosure of information about operations

of individual companies. This suppression occurs when a

small number of companies, usually one or two, account

for a large percentage of the R&D funds or of scientists

and engineers in a particular data cell. The tables most

often affected by this rule are those that contain data

on Federal support to companies for R&D performance.

"(S)," which indicates that the imputation rate—the

percentage of the statistic not reported by respondents

and consequently estimated by Census—exceeds 50 percent

for that item. See table B-3 in Section B for imputation

rates for specific items.

Although publication of data may be withheld from certain

cells, these data are always included in totals. In some

instances data withheld because of high imputation rates

(footnote reference "(S)") can be derived by subtraction

from higher level totals. In such cases the user should be

aware that the derived numbers are statistically unreliable.

In no instance can data be derived that would disclose

operations of individual companies (footnote "(D)").

List of Tables

Table Page

Total Funds for Research and Development

A-1. Trends in industrial R&D performance, by

source of funds: 1953-91 and

estimates for 1992-93... 9

A-2. Selected data for R&D-performing companies,

by industry: 1990-91... 10

A-3. Total (company, Federal, and other) funds

for industrial R&D performance,

by industry and size of company: 1981-91... 12

A-4. Total (company, Federal, and other) funds

for industrial R&D performance,

by industry and size of company: 1991... 14

A-5. Total (company, Federal, and other) funds

for industrial R&D performance,

by industry, size of company, and size of

R&D program: 1991... 15

A-6. Number of R&D-performing companies, by

industry, size of company, and

size of R&D program: 1991... 16

Company and Other (Except Federal) Funds for

Research and Development

A-7. Company and other (except Federal) funds for

industrial R&D performance,

by industry and size of company: 1981-91... 17

A-8. Company and other (except Federal) funds for

industrial R&D performance,

by industry and size of company: 1991... 19

A-9. Company-financed R&D contracted to outside

organizations, by industry and

size of company: 1991... 20

A-10.Number of companies contracting R&D

performance to outside organizations

and total number of R&D-performing

companies, by industry and size of

company: 1991... 21

A-11.Company-financed R&D performed outside the

United States by U.S. domestic

companies and their foreign subsidiaries, by

selected industry: 1981-91... 22

Federal Funds for Research and Development

A-12. Federal funds for industrial R&D

performance, by industry and size of company:

1981-91 .... 23

A-13. Federal funds for industrial R&D

performance, by industry and size of company:

1991... 25

A-14. Federal funds for industrial R&D

performance, by agency and selected industry in

selected years: 1974-91 ... 26

A-15. Number of R&D-performing companies reporting

Federal R&D funds and total

number of R&D -performing companies, by

industry and size of company: 1991... 27

A-16. Industry-administered federally funded R&D

centers (FFRDCs)-- funds by

character of work, full-time-equivalent

(FTE) R&D scientists and engineers,

and total number of employees: 1989-91... 28

R&D Funds by Size of Company and

as a Percent of Net Sales

A-17. Number of R&D-performing companies in

manufacturing and nonmanufacturing

industries, by size of company: 1991... 29

A-18. Concentration of total, Federal, and company

and other R&D funds, and net

sales of R&D-performing companies by size of

R&D program: 1981-91... 30

A-19. Domestic net sales of R&D-performing

companies, by industry and size of

company 1990-91... 31

A-20. Total (company, Federal, and other) R&D

funds as a percent of net sales in

R&D-performing manufacturing companies, by

industry and size of company:

1981-91... 33

A-21. Company and other (except Federal) R&D funds

as a percent of net sales in

R&D-performing manufacturing companies, by

industry and size of company:

1981-91... 35

A-22. Total (company, Federal, and other) R&D

funds as a percent of net sales in

R&D-performing manufacturing companies, by

industry and ranked by size of

R&D program: 1991... 36

A-23. Company and other (except Federal) R&D funds

as a percent of net sales in

R&D-performing manufacturing companies, by

industry and ranked by size of

company-financed program: 1991... 37

A-24. Federal R&D funds as a percent of net sales

in R&D-performing manufacturing

companies, by industry and ranked by size of

Federal R&D program: 1991... 38

Funds for Basic Research, Applied Research,

and Development

A-25. Total (company, Federal, and other) funds

for performance of basic research,

applied research, and development, in

current and in constant dollars: 1953-91... 39

A-26. Funds for performance of basic research,

applied research, and development, by

industry and source of funds: 1989... 40

A-27. Funds for performance of basic research,

applied research, and development,

by industry and source of funds: 1990... 42

A-28. Funds for performance of basic research,

applied research, and development,

by industry and source of funds: 1991... 44

A-29. Number of R&D-performing companies

conducting basic research, by

industry and size of company: 1991... 46

A-30. Funds for industrial applied R&D

performance, by product field and source

of funds: 1991... 47

A-31. Total (company, Federal, and other) funds

for industrial applied R&D

performance, by industry and product field:

1991... 48

A-32. Total (company, Federal, and other) funds

for industrial applied R&D

performance, by industry and secondary

product field: 1991... 50

Energy and Pollution Abatement Research

and Development

A-33. Total (company, Federal, and other) funds

for industrial energy R&D

performance, by industry: 1981-91 and

projected 1992... 55

A-34. Company and other (except Federal) funds for

industrial energy R&D

performance, by selected industry: 1981-91

and projected 1992... 56

A-35. Federal funds for industrial energy R&D

performance, by selected industry:

1981-91 and projected 1992... 57

A-36. Total (company, Federal, and other) funds

for industrial energy R&D

performance, by primary energy source:

1981-91 and projected 1992 ... 58

A-37. Company and other (except Federal) funds for

industrial energy R&D

performance, by primary energy source:

1981-91 and projected 1992 ... 59

A-38. Federal funds for industrial energy R&D

performance, by primary energy

source: 1981-91 and projected 1992... 60

A-39. Number of companies performing energy

research and development, by industry

and size of company: 1991... 61

A-40. Total (company, Federal, and other) funds

for industrial pollution-abatement

R&D performance and

total number of R&D performing companies, by industry:

1981-91 and projected 1992 ... 62

A-41. Company and other (except Federal) funds for

industrial pollution-abatement

R&D performance, by

selected industry: 1981-91 and projected 1992 ... 63

A-42. Federal funds for industrial

pollution-abatement R&D performance, by selected

industry: 1981-91 and projected 1992... 64

A-43. Total (company, Federal, and other) funds

for industrial pollution-abatement R&D

performance, by type of pollution: 1981-91

and projected 1992 ... 65

A-44. Total (company, Federal, and other) funds

for industrial pollution-abatement R&D

performance, by type of pollution and

source of funds: 1990-91 and projected

1992... 66

Geographic Distribution and Distribution by Type

of Cost

A-45. Geographic distribution of total (company,

Federal, and other) funds for industrial

R&D performance in selected years: 1974-91... 67

A-46. Geographic distribution of total (company,

Federal, and other) funds for industrial

R&D performance, by source of funds: 1991... 69

A-47. Distribution of total (company, Federal, and

other) funds for industrial R&D

performance, by industry and type of cost:

1991... 70

Employment

A-48. Number of full-time-equivalent (FTE) R&D

scientists and engineers in R&D-

performing companies, by industry and size

of company: 1982-92... 71

A-49. Number of full-time-equivalent (FTE) R&D

scientists and engineers in R&D-

performing companies, by industry and source

of R&D funds: January 1992... 73

A-50. Cost per R&D scientist or engineer in R&D-

performing companies, by industry

and size of company: 1981-91... 74

A-51. Cost per R&D scientist or engineer, by

industry and size of company: 1991... 76

A-52. Cost per R&D scientist or engineer in

companies ranked by size of R&D

program: 1981-91... 77

A-53. Domestic employment of R&D-performing

companies, by industry and size of

company: 1990-91... 78

A-54. R&D scientists and engineers per 1,000

employees in manufacturing companies,

by industry and size of company: 1981-91... 80

A-55. R&D funds per employee, by size of company:

1981-91... 82

A-56. Concentration of total employment in

R&D-performing companies ranked by

size of R&D program: 1985-91... 83

B-1. Number of companies in the universe, sample, and 1991 panel...

B-2. Standard error of estimate (percentage) of total (company, Federal,

and other) fundsfor industrial R&D performance for all company size groups

and for companies with fewer than 1,000 employees, by industry: 1991...

B-3. Imputation rates for selected items, by industry: 1991...

B-4. Funds for performance of basic research, applied research, and

development, by industry and source of funds: 1989...

B-5. Funds for performance of basic research, applied research, and

development, by industry and source of funds: 1990 ...

B-6. Funds for performance of basic research, applied research, and

development by industry and source of funds: 1991...

B-7. Unit response rates: Number of companies responding to survey,

by industry: 1991...

B-8. Item response rates: 1991...

Section B. Technical Notes

Introduction

The Survey of Research and Development in Industry

provides national estimates of the total expenditures on

research and development (R&D) performed within the United

States by industrial firms, whether U.S. or foreign owned.

It is a sample survey in which all R&D-performing companies,

including privately held firms, are intended to be included

or represented. All companies that are identified in the

sample year as spending more than $1 million annually on R&D

in the United States receive a survey form every year.

Information from individual companies in the sample is used

to develop national estimates on an industry-by-industry

basis.

The National Science Foundation (NSF) has been sponsoring

a survey of industrial R&D since 1953. The 1991 Survey of

Research and Development in Industry is the 35th in the

annual series sponsored by NSF and conducted since 1957 by

the Bureau of the Census. NSF's Division of Science

Resources Studies monitors the survey. NSF also sponsored

two industrial R&D surveys covering the 1953-56 period that

were conducted by the U.S. Department of Labor, Bureau of

Labor Statistics (BLS).1 Data obtained in the earlier BLS

surveys are not directly comparable with Census figures for

1957-91, because of methodological and other differences.

Respondents receive detailed definitions to help them

determine which expenses to include or exclude from the R&D

data they provide. Nevertheless, the statistics presented

in this report are subject to response and concept errors

caused by different respondent interpretations of the

definitions of R&D activities and by variations in company

accounting procedures. Consequently, the data are better

indicators of changes in, rather than absolute levels of,

R&D spending and personnel. Data quality has improved

substantially since the early surveys were conducted, mainly

because respondents have adopted more accurate and

sophisticated accounting procedures over the years.

The survey's primary focus is on U.S. industry as a

performer of, rather than as a source of funds for, R&D.

Thus, data on Federal support of R&D activities performed by

industry are collected and appear in several tables, but

data on industrial funding of R&D undertaken at universities

and colleges and other nonprofit organizations are not

included in the major tables.2

The survey data provide (1) national estimates of total

R&D performed by industry in the United States; (2) the

portion of the effort that is financed by U.S. Government

funds; and (3) the amount financed by the companies

themselves or by other non-Federal sources such as State and

local governments or other industrial firms. Also included

are data on both the number of employees and the number of

R&D-performing scientists and engineers at the firm, as well

as on the domestic net sales for the company, and on the

total funds for R&D financed by the domestic firm but

performed outside the United States. The tables developed

from the survey data are in section A, and copies of the

1991 survey questionnaire, instructions, and other survey

documents are reproduced in section C.

The scope of the survey has been expanded and refined over

the years in response to an increasing policy need for more

detailed information on the nation's R&D effort. For

example, questions on energy R&D were added in the early

seventies, following the first oil-shortage crisis. On the

other hand, the frequency of collection of certain data

items has been reduced in an attempt to alleviate some of

the respondent burden that has been placed on industry from

all sources in recent years. Since the 1978 survey, a

detailed questionnaire, Form RD-1, has been used only to

collect data for odd-numbered years and an abbreviated

version, containing only the most crucial data elements, has

been used to collect data for the intervening, even-numbered

years. Questions appearing only on the long form, request

detail on: R&D by product field; company expenditures for

R&D projects that were contracted to outside organizations,

rather than performed in-house; Federal R&D support to the

firm, by contracting agency; R&D expenditures by geographic

area; and some detailed data on energy and pollution-

abatement R&D activities. This report provides data

collected from the long form.

Survey Methodology3

Overview

Data in this survey are based on a sample of industrial

firms, selected approximately every 5 years (e.g., 1976,

1981, 1987). In intervening years only a subset of the

sample, or panel, receives annual survey forms and Census

makes estimates for the changes in R&D for the firms not

canvassed annually. The sampling unit for this survey is

the enterprise, or company, defined as a business

organization consisting of one or more establishments under

common ownership or control. The 1991 data in this

publication are based on responses from a panel derived from

the sample selected and first used for survey year 1987

(table B-1).

The Standard Statistical Establishment List (SSEL), a

Census compilation that contains information on 3.5 to 4.0

million establishments, was the universe frame used to

select the 1987 sampling frame. Where necessary for

multiestablishment companies, Census summed establishment-

level data to the company level and then assigned a single

Standard Industrial Classification (SIC) code to that

firm—the SIC code of the establishment(s) having the highest

dollar-value of payroll.4 The frame from which the sample

was drawn includes companies in all manufacturing industries

and, on the basis of earlier samples, a select number of

nonmanufacturing industries known to conduct R&D.

The weight given to an individual company is, in general,

the inverse of the probability that the company would have

been selected for inclusion in the sample. Certainty

companies have a weight of 1.00. The company weight is

retained both in the sampling year and in the years between

samples. To minimize respondent burden, only the panel

subset of the sample is canvassed between sample years.

Most small firms are not re-contacted, but, in each

succeeding year, Census estimates the data for each

noncanvassed firm on the basis of the changes in the initial

value of the R&D reported by the firm and the average growth

rate for the firm's industry.

Frame Creation

In selecting a frame from which to draw a sample of R&D-

performing industries, NSF staff, given a finite budget for

sample selection, make certain assumptions about industries

to maximize sampling efficiency. That is, they apply a

priori knowledge of industrial R&D activity to eliminate

some industries from the possibility of selection by the

initial SSEL sort and to designate other companies for

sampling with 100-percent certainty. In addition, NSF staff

made several innovations for the sample drawn in 1987 to

improve its quality.

As in previous sample years, all companies that had been

on the previous panel received a survey form. In addition,

Census staff reviewed lists of R&D contractors published by

the Department of Defense (DoD) and the National Aeronautics

and Space Administration (NASA) to ensure that all their

large industrial R&D-performing contractors were included in

the sample with certainty.

From the outset, in the latest sample selection the major

goal was to eliminate from the sampling frame to the

greatest extent possible companies unlikely to have R&D

programs. These companies were eliminated to minimize the

number of sample companies chosen thathad no R&D activity.

To accomplish this objective, two steps were taken:

1. NSF staff narrowed the list of nonmanufacturing

industries considered in-scope by eliminating those likely

to engage in little or no R&D activity. Thus, companies in

the eliminated nonmanufacturing industries had no chance of

being selected. This method gave companies in the

manufacturing industries and the remaining nonmanufacturing

industries a greater probability of selection than they had

in the past.

2. Companies with more than 500 employees in the in-scope

industries were sampled with certainty. Companies with less

than 500 employees were subjected to or eliminated from

sampling on the basis of varying employment size level

cutoffs. An assumption was made that in some industries

companies with only a small number of employees are unlikely

to engage in R&D activity, and an employment cutoff was set

for each industry group. Generally, the cutoff was 250

employees with some exceptions (e.g., the cutoff for

hospitals was 1,000 employees). Those companies falling

below the cutoff were eliminated from the universe frame.

To improve coverage of R&D-performing companies, NSF staff

in 1987 provided to Census the names of firms, identified

through media reports or other sources as R&D performers,

that were to be included in the sample with certainty. Most

of these companies, because they met other established

criteria, would have received questionnaires anyway, but a

few were added to the sample by this effort. All certainty

companies—those on lists provided by NSF staff, on lists of

DoD and NASA contractors, companies with more than 500

employees, and previous panel members—are self representing,

i.e., they have sampling weights of unity.

Based on (1) SIC code, (2) total employment cutoffs, (3)

inclusion on an NSF, DoD, or NASA list, or (4) previous

panel membership, approximately 154,000 companies were

identified as in scope of the survey and, therefore were

included in the sampling frame. The new efforts to improve

targeted coverage resulted in a sharp reduction in the size

of the total in-scope universe from about 450,000 companies

in 1981.

It is likely that a small number of companies actually

engaged in R&D activity were omitted from the sampling frame

as a result of these sample selection operations. It was

agreed, however, that the benefit from the new

operations—greater sampling efficiency, resulting in

improved national estimates of industrial R&D expenditures

and employment—far outweighed the cost of the loss of a few

companies that may have been eliminated from the in-scope

sampling frame.

Probability Proportionate to Size

As with most types of economic surveys, the sample

selection process for the industrial R&D survey used

probabilities proportionate to size. That is, large

companies have a proportionately higher probability of

selection than do small companies, where large or small is

measured relative to the statistic being estimated.

For the R&D survey, it would have been ideal if size could

be determined by the amount of a company's R&D expenditures.

Unfortunately, except for the companies that were in the

current panel, it was impossible for Census to know the R&D

expenditure values for firms in the universe frame. One

logical solution was to estimate each company's R&D

expenditures and base the probability of selection on the

estimated values. This strategy had been employed in the

1981 sampling operation.

Clearly, this strategy has weaknesses. Even with the

reduced number of in-scope industries in the sampling frame,

many companies chosen for the frame may not have engaged in

any R&D activity. Nevertheless, the procedure used to

estimate the size of companies was forced to treat all

companies as if they did in fact perform R&D.

Census estimated the size of each company's R&D

expenditures by using a relationship linking the size of

each company's employment to its amount of R&D expenditures.

[Since company employment was known for the universe, it was

possible to use this relationship to estimate R&D

expenditures values for all companies in the frame. Census

derived this relationship for each SIC classification

category, using data collected in the then most recent

(1985) R&D survey]. Rather than treating all companies

equally, the larger the number of employees in a company,

the higher the probability of selection for inclusion in the

sample. It was deemed reasonable to assume that large

companies were more likely to have R&D programs than were

small companies.

One further adjustment was introduced to the sample

selection process. This was based on the assumption that

multiestablishment companies, on average, would be expected

to perform more R&D than single-establishment companies of

the same size and in the same industry. The 1985 panel data

were used to develop this adjustment factor. It should be

noted that, for companies that were in the previous panel,

the actual reported R&D activity was used and the data were

not adjusted.

Sample Allocation and Relative Standard Error Constraints

The sampling program utilized for this operation allowed

parameters to be assigned permitting the sample to be

allocated across various levels, or strata, that correspond

to industry groupings. This procedure permitted a desired

sample size, or a desired sampling error, to be achieved for

each stratum. Estimated errors of total R&D estimates for

these strata were not to exceed certain levels. Since the

amount of funds provided by NSF determined the size of the

sample to be drawn, the only constraint in achieving these

results was that the total sample size across all the strata

could not exceed 12,000-13,000 companies. NSF staff

provided relative rankings for each industry group—high,

medium, or low—to determine the precision of the estimate.

An actual translation to what high, medium, or low meant,

specifically, could not be determined until Census staff

arbitrarily investigated several sampling error levels,

computed the sample size that these levels implied, and

applied the constraint of the total sample size of 13,000.

The result of this investigation led to the following

criteria for the target sampling error of estimate of funds

for R&D performance:

a. High precision: sampling error not to

exceed 2 percent

b. Medium precision: sampling error not to

exceed 5 percent

c. Low precision: sampling error not to

exceed 10 percent

Based on the desired precision, these criteria suggested a

total sample size of approximately 13,500. This number was

not excessively beyond the stated limit of 13,000, so this

sample size parameter was chosen for the selection process.

One limitation should be noted. Sampling errors were

controlled by using a universe total that, in large part,

was improvised; that is, and as previously noted, Census

assigned an R&D value to every company in the frame, even

though many of these companies may not actually have had R&D

expenditures. The value assigned was imputed for the great

majority of companies in the frame, and, as a consequence,

the estimated universe and the distribution of individual

company values did not necessarily reflect reality.

Estimates of sampling variability were nevertheless based on

this distribution. The presumption was—and this had been

confirmed in the previous sample selection—-that actual

variation in the sample design would be less than that

estimated, because many of the sampled companies have true

R&D values of zero, not the widely varying values that were

imputed using total employment as a predictor of R&D. Thus,

the 2-percent, 5-percent, and 10-percent error levels

described earlier are conservative. (See table B-2 for a

complete list of the actual standard errors in the 1991

survey.)

The particular sample selected is one of a large number of

the same type and size that, by chance, might have been

selected. Estimates from each of the different samples would

differ somewhat from each other and from the results of a

complete canvass conducted under essentially the same

conditions as the survey. In addition to sampling error the

estimates are subject to nonsampling error that would also

occur if a complete canvass were to be conducted under the

same conditions.

Sample Selection

The sample selection program was run with a specified

expected sample size of 13,500 and with other parameters set

to ensure compliance with the relative standard error

constraints. An actual sample of 13,917 was selected. The

actual sample size differs from the specified for two

reasons. First, the selection program used independent

sampling. Each company had an independent chance of

selection, based on its assigned probability; the selection

of a company was completely independent of the selection of

any other company. In independent sampling, sample size

itself is a random variable. Theoretically, a sample of

size zero or a sample the size of the entire universe is

possible, but the probabilities of these extremes are so

small that these are nearly impossible situations. The

actual sample size is usually quite close to the specified

size. If there is too much deviation, the program is simply

executed again.

Second, a minimum probability rule was imposed. As noted

earlier, the sampling program assigns probabilities

proportionate to size (where size in this case is the

imputed R&D value assigned each company). Selected

companies that have R&D programs vastly larger than their

assigned values can have adverse effects on the estimates

once the data are collected. To lessen these effects, the

maximum weight a company can assume was arbitrarily

controlled by specifying that the probability of selection

cannot be less than a certain value. If the probability,

based on its size, is less than this minimum value, then it

is set equal to this value. The consequence of raising

these original probabilities to the minimum probability is

to raise the expected sample size. It is likely that most

of the difference between the specified sample size and the

actual sample size is due to this rule.

The Annual Panel

The panel is a group of companies that receive a survey

questionnaire, form RD-1, annually. The following is a

description of how the present panel was formed from the new

sample in 1987.

The basic tool for the survey is form RD-1, which is used

to collect detailed R&D information. The 1,095 companies

that were in the old panel and had received a 1986 RD-1 form

received an RD-1 form for 1987. The remaining certainty

companies (6,903) and other companies (5,919) in the new

sample received an RD-1A survey form for 1987. Form RD-1A

is an abbreviated version of RD-1 and generally is mailed to

companies only in the year in which a new sample is drawn.

The purpose is to canvass smaller R&D performers but to

impose a minimum of reporting burden on them.

Of the 13,917 companies that received a form in 1987,

3,648 respondents reported that their companies were R&D

performers or were identified as R&D performers based on

information from other sources. The 3,648 companies were

ranked by total R&D funds (both Federal and company and

other) within each SIC code. All companies with over $1

million in total R&D expenditures were placed on the RD-1

panel. In some industries Census added to the panel

companies with less than $1 million in R&D expenditures in

an attempt to ensure 95-percent coverage of the R&D total

for each industry.

The result of this process was an increase of panel size

from 1,095 companies based on the previous sample to 1,742

companies based on the sample drawn in 1987. All companies

selected for the panel will receive the RD-1 questionnaire

annually until the next sample is drawn. The other

companies will not receive another questionnaire; their R&D

data will be estimated, in subsequent years, by Census

staff, using their 1987 reports as a base.

Followup for Survey Nonresponse

The 1991 survey questionnaires were mailed in March 1992,

and recipients were asked to respond within 60 days. Thirty

days later, letters were mailed to all survey recipients,

reminding them that their completed questionnaire was due

within the next 30 days. At 60 days, an initial set of

followup letters was sent to all nonresponding firms. Two

additional followup mailings were made to persistent

nonrespondents, at 90 and 120 days. The 90-day followup

mailing included a replacement questionnaire.

In addition to the followup mailings, telephone followup

was used to encourage response among those nonresponding

firms ranked among the 200 largest R&D performers, based on

1989 expenditures. Telephone followup was also used for

these firms during the initial data edit phase of survey

operations.

Imputation for Item Nonresponse

Even when forms are returned, they are often incomplete.

Many firms choose to return the survey forms with one or

more items blank, for varying reasons—for instance, the

internal accounting procedures of the firm may not allow it

to quantify R&D by pollution-abatement expenditures. In

addition, some firms, as a matter of policy, refuse to

answer any voluntary questions. Four data items on the

questionnaires (total R&D expenditures, Federal R&D funds,

net sales, and total employment), however, are included in

Census' mandatory statistical program. The response rate

for these mandatory items was significantly higher than for

the other items on the survey.

When respondents do not provide the requested information,

Census estimates the missing data for each firm, using an

imputation algorithm. In general, the imputation algorithms

employed by Census compute values for missing items by

applying the average percentage change for the target item

in the nonresponding firm's industry to the item's prior

year value for that firm, reported or imputed. This

approach, with minor variation, is used for most items on

the survey form. Note that all items for nonpanel

firms—that is, firms that are surveyed in the sample year,

but that, because of size, are not surveyed again until a

new sample is selected—are estimated in the nonsample years,

using these imputation processes.

Table B-3 contains imputation rates for the principal

survey items. For a detailed discussion of the problems

stemming from item nonresponse, see the NSF technical

report, Estimating Basic and Applied Research and

Development in Industry: A Preliminary Review of Survey

Procedures, NSF 90-322.

There are two survey items that do not use this general

imputation methodology: (1) geographic distribution of R&D

expenditures and (2) R&D expenditures distributed by basic

research, applied research, and development (character of

work). The imputation method for computing estimates of the

geographic distribution of R&D expenditures was revised in

1989 to include the use of secondary source information as

well as focused telephone followup. The secondary source

used is the Directory of American Research and Technology

1991, R.R. Bowker, New York, which lists the locations of

R&D laboratories for over 6,700 firms in the U.S. R&D

expenditures for nonresponding firms listed in this

directory as having laboratories in only a single State were

assigned to that State. The remaining nonresponding firms

were contacted by telephone and asked to distribute their

reported R&D expenditures. Most were able to provide this

information. This modified approach is used only for

nonresponding firms ranked among the 200 largest R&D

performers. The imputation rates listed in table B-3

reflect gains made using this method.

The imputation procedure used for the basic research,

applied research, and development expenditure estimates was

revised in 1987 for use with 1986 and later data and also

differs from the general imputation approach. Response to

character-of-work items has been declining in recent years

and, as a result, imputation rates have been increasing.

The general imputation procedure became increasingly

dependent upon information that had been imputed in prior

years, thereby distancing current-year distribution

estimates from the time they were reported. In addition, to

reduce the reporting burden of this survey, smaller,

nonpanel firms in the sample have never been asked to

disaggregate their R&D expenditures into character-of-work

categories; consequently, imputation is required for all

such firms.

Because of the increasing dependence on imputed data, NSF

chose not to publish estimates of industrial basic research,

applied research, and development in this report series,

beginning in 1986. In 1987, Census revised its approach for

imputing character-of-work estimates. The revised method

calculates the character-of-work distribution for a

nonresponding firm only if that firm reported a distribution

within a 5-year time period, extending from 2 years before

to 2 years after the year requiring imputation. Imputation

for a given year is initially performed in the year the data

are collected and is based on a character-of-work

distribution reported in either of the 2 previous years, if

any. It is again performed using new data collected in the

next 2 years. Thus, character-of-work estimates are revised

as newly reported information becomes available and are not

final for 2 years following their initial publication. This

revised imputation procedure was first used for the 1986

estimates.

If reported data are not available for a firm, Census does

not impute a character-of-work distribution for that firm.

As a consequence of this restriction, only approximately 70

percent of the total estimated R&D expenditures were

distributed across these categories in the 1986 Census

estimates. Those expenditures not meeting the requirements

of the new imputation methodology were placed in the newly

created "not distributed" category. Tables B-4 through B-6

show the current character-of-work estimates provided by

Census along with the "not distributed" component for 1989,

1990, and 1991.

However, NSF's objective in this survey is to provide

estimates for the entire population of firms performing R&D

in the United States. Because the revised Census imputation

procedure no longer produces such estimates because of the

"not distributed" component, this objective was not being

met for character-of-work estimates. A separate NSF

technical report, Estimating Basic and Applied Research and

Development in Industry: A Preliminary Review of Survey

Procedures, NSF 90-322, presents character-of-work estimates

for 1986, 1987, and 1988 and examines several alternative

approaches to apportioning the "not distributed" data to the

basic research, applied research, and development

categories. The tables in the report illustrate how

estimated figures for basic research, applied research, and

development can vary, depending on assumptions made about

the R&D expenditures that are not distributed. The 1989,

1990, and 1991 estimates of basic research, applied

research, and development provided in section A of this

report were calculated using the baseline estimation method

presented in NSF 90-322. In the baseline estimation method,

the "not distributed" expenditure data are allocated, by

industry group, to basic research, applied research, and

development categories, using the percentage splits in the

distributed category for that industry. The allocation is

done at the lowest level of published industry detail only;

higher levels are derived by aggregation.

Response Rates and Mandatory Versus Voluntary Reporting

Detailed unit and item response rates for the 1991 survey

are shown in tables B-7 and B-8, respectively.

Current survey reporting requirements divide survey items

into two groups: mandatory and voluntary. Response to four

data items on the questionnaires (total R&D expenditures,

Federal R&D funds, net sales, and

total employment) is mandatory, whereas response to the

remaining items is voluntary. During the 1990 survey cycle,

NSF conducted a test of the effect of reporting on a

completely voluntary basis to determine if combining both

mandatory and voluntary items on one questionnaire

influences response rates. For this test the 1990 sample

was divided into two panels of approximately equal size.

One panel, the mandatory panel, was asked to report as usual

(four mandatory items and the remainder voluntary), and the

other panel, the voluntary panel, was asked to report all

items on a completely voluntary basis. The result of the

test was a decrease in the overall survey response rate to

80 percent from levels of 88 percent in 1989 and 89 percent

in 1988. The response rates for the mandatory and voluntary

panels were 89 percent and 69 percent, respectively.

Detailed results of the test were published in Research and

Development in Industry: 1990 (NSF 94-304).

Comparability of Statistics

Several procedures have been undertaken to revise data and

maintain the reliability of the time series. Revisions

affecting the comparability of current-year with most recent

prior year statistics and historical statistics are

explained below.

Two-Year Comparability

Before the survey forms are mailed each year, Census

imprints the data reported by the respondent the previous

year (or 2 years earlier for items asked only in odd-

numbered years). Respondents are asked to adjust data for

the previous year(s), if necessary, to make them comparable

to data provided for the current year. Such adjustments are

necessitated by any of several developments, e.g., changes

in company structure such as mergers, acquisitions, or

divestitures; changes in reporting concepts; and even

correction of any errors that may have been made previously.

Thus, there is comparability in the reporting of survey data

over any 2-year period.

These adjustments can be examined by comparing data for

the same year reported in two succeeding periods, e.g., 1990

data appearing in Research and Development in Industry: 1990

may differ from 1990 data in this volume. Totals for broad

classifications are likely to be very close in the two

editions; differences may be more noticeable in the finer

detail. These differences underscore the point that the

measures are approximate and indicative rather than precise.

Historical Revisions

The published industry R&D survey data are revised

periodically, usually because of changes in the SIC code

classifications of companies. In addition, two other major

data revisions were made in the data for the 1981-87

periods.

Companies may shift from one SIC code into another

because of (1) the growth or decline of product lines; (2)

the merger of two or more companies; (3) the acquisition of

one company by another; (4) divestiture; or (5) the

formation of conglomerates. When Census is aware of the

year in which changes (2), (3), (4), or (5) occurred

(respondents are asked on the questionnaire about changes in

ownership), the firm's data are reclassified into the new

industry for the year the change actually occurred. If a

change is not discovered until a new sample is selected or

if it cannot be determined when a change actually occurred

(i.e., (1)), other methodologies are used to adjust, or

backcast, the data to reflect the move of a company out of

one industry and into another. Since 1967, three revisions

were made in the data, covering the periods 1967-76, 1976-

81, and 1981-87.

1. The 1967-76 Period

The SIC codes assigned to companies in the panel for the

years 1967 through 1975 were based on data reported in the

1967 economic census. The 1974 Standard Statistical

Establishment List file was used to assign SIC codes to

companies in the next panel chosen for the 1976 survey.

These codes were used to revise 1975 data received in the

1976 survey. The SIC codes of companies in the 1967 and

1976 panels were examined to determine which companies had

changed classifications. Since it was not known in which

year changes actually occurred, data for companies that had

changed SIC codes were revised for the years 1968 through

1974, to smooth the changes over the period 1967-76. To

illustrate, if a company was originally in SIC A in 1967 but

was discovered to be in SIC B in 1974, its data for 1967-74

were allocated between the two industries as follows: for

1967, all of the company's data were retained in industry A;

for 1968, 14.3 percent of the company's data were allocated

to industry B, and the remainder retained in industry A; for

1969, 28.6 percent were allocated to industry B, and the

remainder retained in industry A; and so on until 1974, when

all of the company's data were allocated to industry B.

2. The 1976-81 Period

Backcasting revisions in the industry R&D data for 1976-80

were also made to reflect the classification changes

revealed in the new sample drawn in 1981, however, a

different methodology was used for the revisions.

When the new panel was selected in 1981, companies were

assigned SIC codes from Census' 1981 SSEL and the 1981

Enterprise Files. Prior-period (1980) data were collected

in the 1981 survey. These data were presumed to be more

accurate than those collected in the 1980 survey, because

they reflected not only updated SIC code classifications but

were obtained from a larger panel, providing better coverage

of U.S. industry. Data obtained for 1976-80, using the

panel selected in 1976, were revised subject to the

following constraints:

a. Data for 1976, revised from the 1977 survey,

would remain unchanged, since this was the first

year of the 1976 panel and since that panel was an

accurate reflection of company SIC codes in 1976.

b. Data for 1980 collected in the 1981 survey

would be used instead of the 1980 data collected

in the 1980 survey.

An algorithm was used to link data from 1976 with those

collected in 1981, preserving to the greatest extent

possible year-to-year trends in data for each industry by

revising data for the years 1977, 1978, and 1979.

The following data elements were adjusted, using the

methodologies just described: Funds spent on R&D (total,

Federal, company and other, and R&D performed outside the

United States); number of full-time equivalent (FTE) R&D

scientists and engineers; total and company and other R&D

funds as a percent of net sales; and cost per R&D scientist

or engineer. No adjustments were made in other data

elements.

3. The 1981-86 Period

Similar backcasting revisions in the industry R&D data

were made in 1987 for the industry data previously reported

for the years 1981-86. Several other adjustments were made

in 1987 in addition to the original methodology described

for the revisions made in 1981.

When the 1987 sample was selected, companies were assigned

SIC codes from the 1987 SSEL file. Companies in the sample

were asked to submit their prior-year (1986) data along with

the information collected for the 1987 survey. These new

1986 data were used in lieu of the data originally collected

in the 1986 survey because they reflected updated SIC code

classifications of companies and also were collected from a

much larger sample that was designed to provide better

coverage of U.S. industry, particularly the nonmanufacturing

portion. Data obtained for 1981-86 using the panel selected

in 1981 were revised in 1987, subject to the following

constraints:

a. Data for 1981, revised from the 1982 survey,

would remain unchanged, since this was the first

year the 1981 panel was used, and since that panel

was an accurate reflection of company SIC codes in

that year.

b. Data for 1986 collected in the 1987 survey

would be used instead of the 1986 data collected

in the 1986 survey.

There was an increase of $9.3 million between the 1986

data based on the old panel and the 1986 data reported by

the new panel identified in 1987. Most of that aggregate

revision was caused by the addition of new R&D performing

firms identified through the revised sampling procedures in

1987. An algorithm was used to link data from 1981 with

those collected in 1987, to the greatest extent possible

preserving year-to-year trends that had previously been

reported in data for each industry by revising data for the

years 1982, 1983, 1984, and 1985. Interested persons should

contact Census to obtain further information about the

construction and content of the algorithm.

4. Federal Data for 1986

Data for a merger between two companies were incorrectly

processed in the 1988 tabulations, resulting in an overcount

of Federal R&D support for industry in the amount of $356

million in 1986. The corrected 1986 industry R&D total

(company, Federal, and other funds) is $87,823 million, and

the Federal total is $27,891 million. These corrections are

reflected in the detailed statistical tables in this report.

No revisions to 1988 individual industry data that were

published were necessary, because data for the affected

industry were suppressed to avoid possible disclosure of

information about operations of individual companies (i.e.,

these data cells are marked with the symbol "D").

5. Data Reevaluations for 1987

Many of the small firms identified in the 1987 sample were

receiving this kind of survey questionnaire for the first

time, and some were unaccustomed to reporting financial

information rounded to the nearest thousand dollars.

Several of these firms, therefore, mistakenly reported their

expenditures for R&D in dollars, not thousands of dollars as

requested. Consequently, several firms were selected for

the 1987 panel based on an incorrectly reported size of

their R&D program. Many of these firms realized the error

when they received the 1988 form with the preprinted 1987

data. The respondents corrected their 1987 data when they

submitted the 1988 forms.

Furthermore, for 1988 several of these firms reported that

they had no R&D program, even though they had previously

reported expenditures for R&D activities in the 1987 survey.

After verifying by phone that these firms had no R&D

activity, Census staff declared these firms to be out-of-

scope and blanked the erroneous information from the 1987

estimates during processing of the 1988 survey results.

Similar reporting errors were encountered during the data

collection and processing for the 1989 survey. In addition,

review of the 1989 estimates revealed an error in the

revision procedure used for firms discovered to be out-of-

scope in 1988. This error resulted in only a partial

correction for these firms during the 1988 processing.

These errors caused Census to take the unusual step of

revising the 1987 R&D expenditure estimates during 1989

survey processing. Revisions to other than only previous

year data are rarely done, except during sample years when

historical adjustments are made to account for shifts in

industrial classification.

Survey Definitions5

Company (and other) funds for R&D - cost of R&D actually

performed within the company and funded by the company

itself or by other non-Federal sources via contract not

including the cost of R&D supported by companies but

contracted to outside organizations, such as research

institutions, universities and colleges, nonprofit

organizations, or (to avoid double-counting) other

companies. (Since this is a survey of R&D performers,

industrial firms that undertake R&D supported by outside

organizations do report the funds received in payment for

the R&D work they perform. These moneys are classified

under the industries of the performing companies.)

Cost per R&D scientist or engineer - the arithmetic mean of

the numbers of FTE R&D scientists and engineers reported for

January in 2 consecutive years divided into the total R&D

expenditures of the earlier year, with the ratio attributed

to the earlier year. (For example, the mean of the numbers

of FTE R&D scientists and engineers in January 1991 and

January 1992 is divided into total 1991 R&D expenditures for

a total cost per R&D scientist or engineer in 1991.)

Employment, FTE R&D scientists and engineers - persons

engaged in scientific or engineering work at a level that

requires knowledge of physical, life, engineering, or

mathematical science, equivalent, at least, to that acquired

through completion of a 4-year college program with a major

in one of those fields employed by the company during the

January following the survey year. (The statistics in this

report show the full-time equivalent (FTE) employment. The

FTE is the number of scientists and engineers in the company

who are assigned full-time to R&D projects together with the

fraction of total work time spent on R&D projects by non-

full-time R&D scientists and engineers.)

Employment, total - number of persons domestically employed

by R&D-performing companies in all activities during the pay

period that includes the 12th of March.

Federally funded research and development centers (FFRDCs):

R&D-performing organizations administered by industrial,

educational, or other institutions on a nonprofit basis,

exclusively or substantially financed by the Federal

Government. (FFRDCs are supported by the Federal Government

either to meet a particular R&D objective or, in some

instances, to provide major facilities at universities for

research and associated training purposes. R&D expenditures

of the FFRDCs -administered by industry are included with

the Federal R&D data of the industry classification of each

of the administering firms. The FFRDCs are listed below.)

Supported by the Department of Energy:

Bettis Atomic Power Laboratory Knolls Atomic Power

Laboratory

Westinghouse Electric Corp. General Electric Co

West Mifflin, PA Schenectady, NY

.

Hanford Engineering Development Laboratory Oak Ridge

National Laboratory

Westinghouse-Hanford Corp. Martin Marietta Energy

Systems, Inc.

Richland, WA Oak Ridge, TN

Idaho National Engineering Laboratory Sandia National

Laboratories

EG&G Idaho, Inc.; Western Electric

Company, Inc.—Sandia Corp.

Argonne National Laboratory, West; Albuquerque, NM

Rockwell International Corp.;

Westinghouse Electric Corp. Savannah River

Laboratory

Idaho Falls, IA Westinghouse Electric

Corp.

Aiken, SC

Supported by the National Institutes of Health,

Department of Health and Human Services:

NCI Frederick Cancer Research Facility

Program Resources, Inc.

Frederick, MD

Funds for R&D, Federal - receipts for R&D performed by the

company on Federal R&D contracts or subcontracts and R&D

portions of procurement contracts and subcontracts.

Funds for R&D, total - operating expenses incurred by a

company in the conduct of R&D in its own laboratories or

other company owned or operated facilities including wages

and salaries, materials and supplies consumed, property and

other taxes, maintenance and repairs, depreciation, and an

appropriate share of overhead and excluding capital

expenditures.

Net sales and receipts - dollar values for goods sold, or

services rendered, by R&D-performing companies to customers

(outside the company), including the Federal Government,

less such items as returns, allowances, freight, charges,

and excise taxes. (Domestic intracompany transfers and

sales by foreign subsidiaries are excluded, but transfers to

foreign subsidiaries and export sales to foreign companies

are included.)

Research and development - basic and applied research in the

sciences and engineering and the design and development of

prototypes and processes, excluding quality control, routine

product testing, market research, sales promotion, sales

service, other nontechnological activities or routine

technical services, and research in the social sciences or

psychology.

Basic research - original investigations for the

advancement of scientific knowledge not having specific

immediate commercial objectives, although such

investigations may be in fields of present or potential

interest to the reporting company.

Applied research - investigations for the discovery of

new scientific knowledge having specific commercial

objectives with respect to products or processes.

(Applied research differs from basic research chiefly

in terms of the objectives of the reporting company.)

Development - technical activities not routine in

nature concerned with translating research findings or

other scientific knowledge into products or processes.

(Not included are routine technical services to

customers or other activities excluded above.)

Section C. Survey Documents

Page

National Science Foundation Cover Letter 121

Bureau of the Census Cover Letter 123

Survey Instructions 125

Survey Questionnaire, Form RD-1 133