Executive Summary
The Advisory Committee on Measuring Innovation in the 21st Century Economy was established by the Secretary of
Commerce in September 2006 to recommend ways to improve the measurement of innovation in the economy. In its report, the Advisory Committee outlines its recommendations to the Secretary of Commerce for steps to be taken by the government, the business community, and government and private sector researchers to foster and improve the measurement of innovation in the economy.
The first act of the Advisory Committee was to establish a definition of innovation that would identify what should be measured. The definition adopted by the Committee is:
The design, invention, development and/or implementation of new or altered products, services, processes, systems, organizational structures, or business models for the purpose of creating new value for customers and financial returns for the firm.
The definition recognizes that the innovation to be measured is more than simply something new; it has the added component of adding value for both customers and firms. The definition also recognizes that innovation measurement needs to extend beyond simply measuring inputs. While it is important to track inputs to innovation – such as research and development spending – that is not enough. Outcomes of innovative activity need to be tracked and measured to determine fully the impact of innovation on the economy.
As part of its work, the Committee published a Federal Register request for public comments and received many helpful responses. These comments were extremely useful in informing both the guiding principles and the recommendations adopted by
the Committee. The Federal Register request and copies of all the comments received can be accessed on the Advisory Committee’s web site at www.innovationmetrics.gov.
The following set of principles was developed by the
Advisory Committee to guide its own work:
■ Innovation data collection efforts should build on the way firms assess the effectiveness of their innovative activities.
■ When developing better ways to quantify innovation in the marketplace, consideration should be given to measuring the impact of legislation and regulations on innovation.
■ Because of the nature of innovation and, in particular, the collaborative nature of the innovative process, there needs to be tolerance of qualitative and subjective measures.
■ Innovation measurement should not be static. Measurement is an iterative process that needs to be treated less like a ‘project’ and more
like an ongoing ‘dialogue.’ Learning and improvement are to be gained from each stage of the process.
■ Innovation measures should allow for analysis at the establishment, firm, industry, country, international, and, where possible,
regional levels.
■ A conservative approach should be taken to any new data collection effort by recognizing tradeoffs between costs and potential benefits and considering resource and regulatory constraints. The implementation of pilot projects to gauge the costs and benefits of new data collection efforts is encouraged. To the extent possible, new innovation measures should be able to be ‘back-tested’; that is, if applied to historical data, the measures should produce the expected innovation relationship.
The principles guided the work of the Advisory Committee and should also apply to implementation of the Committee’s recommendations.
The bulk of the Advisory Committee’s work was devoted to developing recommendations to the Secretary of Commerce for actions to improve innovation measurement. The recommendations endorsed by the Committee are summarized briefly below and appear in a more comprehensive list at the end of this summary. All of the recommendations are described in more detail in the body of the report.
WHAT THE GOVERNMENT SHOULD DO
To achieve the long-term goal of measuring the impact of innovation on the economy, the Advisory Committee recommends that the government create a coordinated emphasis on innovation measurement. The effort will require structural refinements of existing government data sets, the collection of new and better data, improved linkages among statistical agency data sets, and expanded data sharing/synchronization authority.
In particular, the Advisory Committee recommends that the government:
■ Create a stronger framework for identifying and measuring innovation in the national economy.
■ Better leverage existing data among the statistical agencies to allow for the consistent estimation of the contributions of innovation in the gross domestic product (GDP) and productivity accounts and to develop greater understanding of innovation.
■ Increase access to data in order to facilitate more robust innovation research.
■ Convene one or more workshops or forums under the auspices of the Secretary of Commerce to discuss innovation drivers, impediments and enablers.
■ Continue participation in the international dialogue related to measuring and analyzing innovation and ensure that U.S. efforts
are internationally compatible to the extent possible.
■ Consider development of a national innovation index when more work has been done on both data collection and analysis of innovation drivers.
■ Support funding necessary to implement the above recommendations.
Most of the recommendations for government action build on existing programs or activities. These include the U.S. National Income and Product Accounts (NIPAs) which constitute the official framework used by the Commerce Department’s Bureau of Economic Analysis (BEA) to estimate output, income and expenditure, trade, capital formation, and wealth in the U.S. economy. The NIPAs were devised to help policymakers deal with the severe economic fluctuations produced by the Great Depression and World War II and have been continually refined since then. The NIPAs now provide policymakers with an unrivaled ability to adjust policy quickly and appropriately in response to short-term economic fluctuations. The NIPAs were not originally designed to measure innovation or delve into the causes of long-term productivity growth. Today, however, when policymakers are increasingly shifting attention from short-term stabilization to long-term economic growth, changes to the NIPAs need to be made to accommodate this new focus.
Refining the framework for measuring the performance of the national economy is an essential element in the government’s program to measure innovation in the national economy and refine overall economic measurement. These improvements will take time as there is considerable preliminary work to be done not only by BEA but also by the other statistical agencies upon which BEA relies for data. The four major elements of this program are: integrating industry-level estimates of total factor productivity1 with the NIPAs; creating a supplemental innovation account; improving service sector data; and improving the measurement of intangibles. Work on each of these four elements has already begun, although additional funding will be required to move forward.
The Advisory Committee also recommends that the statistical agencies pursue an agenda in support of the development of linkages between data sets both to improve data consistency and to provide a richer data base for understanding and explaining innovation. Full implementation of such an agenda would require new legislation. The statistical agencies are currently limited in their authority to share data and that, in turn, affects data consistency.
Linkages between establishment-based data and firm- based data would be particularly useful for tracking and measuring innovation. Most U.S. industry statistics are estimated using establishment-level data as the basic ‘building blocks.’ Such statistics have been very useful for many purposes, as they combine data for establishments that do approximately the same things. However, many firms own or control more than one establishment, and those establishments may be located in different geographic areas and may be producing different kinds of goods or performing different kinds of services, some of which, such as technology licensing transactions, may be of particular importance to the innovation process. Reassembling establishment-level data into firm-based statistics may lead to better innovation measurements. Furthermore, firm- or establishment- level data from one data source can be augmented by matching them to corresponding data from other sources to obtain a more complete picture of innovation. All of these linked data are understood best when also matched over time to create longitudinal data sets. In the longitudinal records, the dynamics of business and innovation begin to emerge.
The Advisory Committee recommends that the Secretary support legislation to enable the statistical agencies to undertake expanded data sharing/synchronization activities. In particular, amending the law to expand access to IRS data to additional statistical agencies for the purposes of reconciling the business lists and designing more effective survey business frames would improve our understanding of the U.S. economy.
To encourage more research by non-government researchers, the Advisory Committee recommends that the government encourage innovation research by making public data more transparent through the use of data-tagging or similar methods of making data more user-friendly and by improving access to data through the creation of more public use data files. Such efforts are being undertaken currently by some agencies and the Committee encourages the expansion of such efforts. In addition, the Advisory Committee also recommends the expansion of non- government researcher access to confidential micro- data, including that on business dynamics, while maintaining high standards for confidentiality.
A major issue raised by many of the Committee members was the need to examine innovation drivers, impediments, and enablers. Since the issue was outside the scope of the Committee’s mandate, it was not fully explored. However, given the importance of the topic, the Committee calls on the Secretary of Commerce to convene one or more workshops or forums to examine innovation drivers, impediments and enablers.
The Advisory Committee recognizes the importance of the international dialogue on innovation and recommends that it be continued and that efforts be made to ensure that new innovation measures allow for analysis across countries.
Finally, the Advisory Committee recommends support for the additional funding that will be necessary to implement the recommendations.
HOW THE BUSINESS COMMUNITY CAN HELP
Measuring innovation must be a collaborative process, and there is much that the business community can do to assist and drive improvements in innovation measurement. In particular, the Advisory Committee recommends that the business community:
■ Create, expand and assess firm and industry- level measures of innovation and develop best practices for innovation management
and accounting.
■ Participate in research activities and, as appropriate, make innovation information available to researchers.
One of the guiding principles endorsed by the Advisory Committee was that innovation data collection efforts should build on the way firms assess the effectiveness of their innovative activities. Individual firms, trade associations and other organizations are important partners in developing and testing innovation measures.
WHERE RESEARCH IS NEEDED
While our understanding of innovation has increased over recent years, much more needs to be learned about innovation and its measurement. Government, business, and academic researchers should undertake research – alone and in collaborative efforts – to understand innovation better. In particular, the Committee recommends exploration of the following research areas:
■ Identification and assessment of innovation outcome measures.
■ Identification of gaps in innovation data and how they might be filled.
■ Analysis of relationships between innovation activities and collaboration, innovation performance and firm performance.
CONCLUSION
The recommendations, if adopted, will go far in setting this nation on a course toward effectively measuring the impact of innovation on the economy. The work is essential to understanding and developing better policies for innovation. The Committee calls on the government, the business community and researchers to work together to improve the understanding and measurement of innovation.
