Contact Us Search: All EPA This Area

• You are here: EPA Home
• Research & Development
• National Center for Environmental Research
• Publications & Proceedings
• Publications & Proceedings
• Bibliometrics
• Bibliometrics Analysis for TSE Grant Publications

Bibliometrics

Bibliometrics Analysis for TSE Grant Publications
August 24, 2004

EPA’s National Center for Environmental Research (NCER) has awarded 91 Technology for a Sustainable Environment (TSE) grants since 1995.¹ Thirty-six (40%) of the grantees have submitted their final reports. For these 36 grants, there were a total of 285 publications in the NCER database, and 124 of those publications have been cited 2,752 times.^2,3

Summary

Many TSE publications are highly cited papers. A review of the citations indicates that 34 (11.9%), 21 (7.4%), and 5 (1.8%) of the 285 publications qualified as highly cited when using the criteria for ISI Essential Science Indicators (ESI) highly cited publications (in the top 1%) in the fields of engineering, materials science, and chemistry, respectively. Six (2.1%) of the highly cited TSE publications fall within the top 0.10% and 4 (1.4%) within the top 0.01% of engineering publications tracked by ESI.

Nearly one-third of the TSE publications are published in very high impact journals, i.e., top 10% of the 5,875 journals monitored by ISI Journal Citation Reports (JCR), as indicated by the impact factor calculated by JCR. ISI reports that impact factor is the primary means employed for determining the prestige of journals. In addition, one-third of the TSE publications are published in journals with an immediacy index that ranks in the top 10% of the 5,867 journals reported by JCR, indicating that a significant number of TSE papers are published in journals that are cited quickly after publication.

The TSE average citation rates⁴ are higher than those reported by ISI Essential Science Indicators for the fields of engineering and materials science, and comparable to those for the field of chemistry. The publication rates appear to be increasing with time (i.e., the grants awarded in 1999 are publishing more papers than those awarded in 1995). The citation frequency⁵ also appears to be increasing with time. Analysis of a representative sample of TSE publications projects a total of 1,046 citations in 2004, which is considerably greater than the average 306 citations per year for the years 1995-2003. This analysis also revealed at least one recent “hot paper” among the TSE publications.

1 TSE grants are funded by both EPA and the National Science Foundation (NSF). This bibliometric analysis applies only to the grants funded by EPA, which is about half of the total number of TSE grants.

2 The 91 TSE grants (including completed and uncompleted) have yielded a total of 372 publications, of which 156 have been cited 2,984 times in the literature.

3 This number of citations is probably lower than the actual number because 60% of the final reports have not yet been received and their publications (and associated citations) are not included in the database.

4 The citation rate is the number of citations for papers published in a given year divided by the number of papers published in that year.

5 The citation frequency is defined as the total number of citations for “all years” (for the TSE publications this period is 1995-2003) divided by the number of years in the “all years” period.

Highly Cited TSE Publications

Analysis of the TSE citations indicates that 34 (11.9%) of the 285 publications qualified as highly cited when using the criteria for ISI Essential Science Indicators (ESI) highly cited publications in the field of engineering. This means that 11.9% of the TSE publications are included in the top 1% of highly cited engineering publications tracked by ESI. Five (1.8%) of the TSE publications are included in the top 1% of highly cited chemistry publications and 21 (7.4%) of the TSE publications are included in the top 1% of highly cited materials science publications tracked by ESI.

Six (2.1%) of the highly cited TSE publications fall within the top 0.10% of highly cited engineering publications tracked by ESI, and 4 (1.4%) fall within the top 0.01% of the highly cited engineering publications tracked by ESI. Four (1.4%) of the TSE publications fall within the top 0.10% of chemistry publications tracked by ESI, and 5 (1.8%) of the TSE publications fall within the top 0.10% of materials science publications tracked by ESI. These results are summarized in Table 1.

Table 1. Summary of Highly Cited Publications Analysis

The four highly cited TSE publications in the top 0.01% of highly cited engineering publications in ESI are:

Savage PE, Gopalan G, Mizan TI, Martino CJ, Brock EE. Reactions at supercritical conditions: fundamentals and applications. AIChE Journal 1995;41:1723-1778.
352 Citations

Patten TE, Matyjaszewski K. Copper(I)-catalyzed atom transfer radical polymerizations. Accounts of Chemical Research 1999;32:895-903.
149 Citations

Blanchard LA, Hancu D, Beckman EJ, Brennecke JF. Green processing using ionic liquids and CO2. Nature 1999;399(6731):28-29.
148 Citations

Huddleston JG, Visser AE, Reichert WR, Willauer HD, Broker GA, Rogers RD. Characterization and comparison of hydrophilic and hydrophobic room temperature ionic liquids incorporating the imidazolium cation. Green Chemistry 2001;3(4):156-164.
125 Citations

Self-Citation Impact

In an attempt to quantify the impact of self-citation on the total number of citations, all citations by the primary author of the cited article were eliminated from the count. Five hundred eighty-four of the 2,752 (21.2%) citations are self-citations by the primary author. This number may be slightly higher than average; ISI reports that self-citations often represent about 13% of the citations that a journal receives.

Publication in High-Impact Journals

ISI Journal Citation Reports (JCR) was used to determine if the TSE grantees are publishing in prestigious journals that have a high impact. Two measures were used to support this analysis. The first was the impact factor, which can be used to provide a gross approximation of the prestige of journals. Impact factor is a measure of the frequency with which the “average article” in a journal has been cited in a particular year or period. The annual JCR impact factor is a ratio between citations and recent citable items published. The impact factor is useful in clarifying the significance of absolute citation frequencies. It eliminates some of the bias of such counts, which favor large journals over small ones, or frequently issued journals over less frequently issued ones, and of older journals over newer ones. The impact factor also excludes self-citations. Table 2 presents a ranking by impact factor of the journals in which two or more TSE articles were published, with the exception of the one article in Nature, which was included because Nature ranks very high among all JCR journals for both impact factor and immediacy index.

Table 2. Journals Ranked by Impact Factor

Eighty-one (28.4%) of the 285 TSE publications were published in very high-impact journals as determined by their impact factor ranking in the top 10% of the 5,875 journals monitored by ISI Journal Citation Reports.

The second measure used in the analysis was the immediacy index, which is the average number of times a journal’s current articles are cited in the current year. This provides some indication of how quickly articles in a particular journal are cited (i.e., “hot” topics). Table 3 presents a ranking by immediacy index of the journals in which 2 or more TSE articles were published, with the exception of the one article in Nature, which was included because Nature ranks very high among all JCR journals for both impact factor and immediacy index.

Table 3. Journals Ranked by Immediacy Index

Ninety-four (33.0%) of the 285 TSE publications were published in journals with an immediacy index that ranked in the top 10% of the 5,867 journals reported by JCR.

Citation Rates

The average citation rates for the TSE grants are above the average citation rates for the fields of engineering and materials science, and comparable to those for the field of chemistry reported by ESI. The average citation rates in ESI are calculated for each year of the 10-year period (1993-2003). This calculation considers the total number of citations from the year of publication to the current year divided by the number of papers in a particular field (ESI tracks 22 fields such as engineering, chemistry, and materials science). ESI also reports an average for the full 10-year period referred to as an “All Years” average. The “All Years” average citation rate for TSE publications for the years of this program (1995-2003) is 26.4 citations/publication, which is greater than the “All Years” (1993-2003) average citation rates reported by ESI for the fields of engineering, chemistry, and materials science. Given the high average citation rate for 1995 (because there are only two 1995 TSE publications and one of them has been cited 352 times), it may be more realistic to use the 7.76 average citation rate for TSE publications published from 1997-2003 to compare to the “all years” average citation rates reported by ESI. This 7.76 average citation rate compares favorably with but still exceeds the 2.88, 7.57, and 3.85 ESI “All Years” average citation rates for engineering, chemistry, and materials science, respectively. These data are presented in Table 4.

Table 4. Average Citation Rates

a Average citation rates for papers published by field, 1993-2003, ESI.
b The average citation rate is higher than would be expected because of the small number of total publications (i.e., 2), and the unusually high number of citations (i.e., 352) for one of the two publications in 1995.
c All years for this average citation rate is 1995-2003. The average is calculated by adding the citation rates for 1995-2003 and dividing by 9 years.
d The average is calculated by adding the rates for the years 1996-2003 and dividing by 8 years.
e The average citation rates for 1996-2003 for the fields of engineering, chemistry, and materials science are calculated by adding the rates for the years 1996-2003 and dividing by 8 years. These rates are not reported in ISI Essential Science Indicators but were derived from the ESI average citation rates reported by year.

Citation Frequency

The citation frequency for TSE publications appears to be increasing with time. Analysis of a representative sample of TSE publications projects a total of 1,046 citations in 2004, which is considerably greater than the average 306 citations per year for the years 1995-2003 (calculated by dividing the total number of citations by the number of years since the first publication [2,752 ÷ 9 = 306]).

Publications Rates

There is some evidence that the publication rates for the TSE grants are increasing with time. In examining trends in publication rates, the rates have increased. For the TSE grants awarded in 1995, 41 total publications resulted from 7 grants (average of 5.9 publications/grant). For the grants awarded in 1997, 123 publications resulted from 12 grants (10.2 publications/grant). Although most of the 1999 grants have not been completed, using the data for the completed grants yields a rate of 13.3 publications/grant.

Hot Papers

Examination of the March-April 2004 citations of 32 of the TSE publications revealed one “hot paper.” The selection of “hot papers” in ISI Essential Science Indicators is based on selecting the top cited papers in different fields, but the time frame for citing and cited papers is much shorter—papers must be no more than 2 years old and cited within a current 2-month time period. Papers are assigned to 2-month periods and thresholds are set for each period and field to select 0.1% of papers.

The “hot paper” identified among the TSE publications sample was published in May 2002, and it was cited five times in the 2-month period from March-April 2004. This exceeds the four-citation threshold for a “hot paper” in the field of engineering, and is slightly less than the six-citation threshold for materials science publications. The “hot paper” identified in this sample was:

Wei CM, Li CJ. Enantioselective direct-addition of terminal alkynes to imines catalyzed by copper(I)pybox complex in water and in toluene. Journal of the American Chemical Society 2002;124(2):5638-5639.

 

Top of page

Local Navigation

• NCER Home
• Basic Information
• Grants & Funding
• Guidance & Frequent Questions
• STAR Grant Recipients
• Fellowships
• Small Business
• Research Centers
• Other Programs
  • Biodiversity
  • CNS
  • EPSCoR
  • ETOP
  • Nanotechnology
  • P3
  • STAA
• Research Results
• Science Topics
  • CBRA
  • Childrens Centers
  • Drinking Water
  • Economics
  • Endocrine Disruptors
  • Global Change
  • Particulate Matter
  • TSE
• Publications & Proceedings
• Calendar

• EPA Home
• Privacy and Security Notice
• Contact Us