From the start
of the Advanced Technology Program in 1990, the Economic
Assessment Office began building an internal system of longitudinal
surveys and sponsoring rigorous and groundbreaking economic
and policy studies, all of which have advanced the understanding
of the process of technology-based innovation. For these
evaluation activities, the National Research Council recognized
ATP for setting “a high standard for assessment involving
both internal and independent external review.”
Evaluation
has provided an objective, analytical and empirical basis
for assessing ATP’s operations and impacts of operations.
Cumulatively, these evaluations highlight the value of applying
multiple evaluation methods to complex problems, building
a body of credible evidence over time that ATP is achieving
its objectives.
An Emerging
Knowledge of ATP
ATP-sponsored
evaluation studies have produced an emerging knowledge of
the overall performance of the program, firm behavior, collaboration,
spillover effects, and interfaces with state and international
technology programs.
Overall ATP Performance
- Benefit-cost
case studies provide evidence that the benefits of the
program far exceed its costs. These studies collectively
attributed to ATP more than $18 billion in expected present
value social benefits from about 40 projects, much greater
than the total amount spent by the program of $2.3 billion
to date, a greater than 8 to 1 return on investment.
- As expected,
not all of the projects are strong performers, but several
years after project end an estimated 10% of completed
ATP-funded projects showed strong progress toward creating
and disseminating knowledge and commercializing projects
and processes, and another 31% also showed substantial
progress. Slightly less than 10% of all funded projects
failed to start or were terminated prior to completion
for a variety of reasons.
Firm/Industry Effects
- ATP funding
is complementary to, not a substitute for, private sources
of R&D funds. Industry would not have undertaken
40% of ATP projects and another 40% would have proceeded
on a much slower scale.
- ATP funding
leverages and accelerates R&D, refocuses R&D
on more technically challenging problems and enabling
platforms of technologies, and fills a significant funding
gap in early-stage technology development. ATP funding
accelerates the R&D cycle for 9 out of 10 companies
and over half are ahead by 1 to 3 years.
Collaboration Effects
- High rates
of collaboration in ATP projects are the result of encouraging
the formation of joint venture and the use of subcontractors
and strategic partners by single applicant companies.
Four out of five ATP projects involved collaborative
relationships, ranging from R&D partnerships with
other firms, universities, and non-profit labs, to alliances
with other firms to pursue commercialization.
- Collaborations
with universities were frequent and they enhanced the
research capabilities of the firms and provided an avenue
of knowledge diffusion from and through the universities.
Spillover Effects
- Considerable
evidence that ATP-funded projects generate outputs—publications,
patents, patent citations, collaborative linkages, and
products—that potentially lead to knowledge and
market spillovers.
- ATP selects
projects with attributes conducive to generating large
knowledge spillover effects. Those attributes included
linkages to other organizations, and a positive attitude
of award winners toward information sharing.
Source:
The framework for presenting these results is from a recent
publication, A
Toolkit for Evaluating Public R&D
Investment: Models, Methods, and Findings from ATP’s
First Decade: Models, Methods, and Findings from ATP’s
First Decade (NIST GCR 03-857). This factsheet updates
the findings in the Toolkit and adds findings for the most
recent 5 years of evaluation work.
Factsheet
1.A1 (April 2005 by Connie Chang) |