Jump to main content.


Bibliometrics

Site Navigation
Archives
Research Project Search
Enter Search Term:

NCER Advanced Search

Bibliometric Analysis for the U.S. Environmental Protection Agency/Office of Research and Development’s Safe Pesticides/Safe Products Research Program
December 2006

This is a bibliometric analysis of the papers prepared by intramural and extramural researchers of the U.S. Environmental Protection Agency’s (EPA) Safe Pesticides/Safe Products Research Program.  For this analysis, 357 papers were reviewed, and they were published from 1996 to 2006. These publications were cited 3,534 times in the journals covered by Thomson’s Web of Science 1 and Scopus 2.  Of these 357 publications, 288 (81%) have been cited at least once in a journal.

Searches of Thomson Scientific’s Web of Science and Scopus were conducted to obtain times cited data for the safe pesticides/safe products journal publications.  The analysis was completed using Thomson’s Essential Science Indicators (ESI) and Journal Citation Reports (JCR) as benchmarks. ESI provides access to a unique and comprehensive compilation of essential science performance statistics and science trends data derived from Thomson’s databases. For influence and impact measures, ESI employs both total citation counts by field and cites per paper scores. The former reveals gross influence while the latter shows weighted influence, also called impact.  JCR is a recognized authority for evaluating journals.  It presents quantifiable statistical data that provide a systematic, objective way to evaluate the world’s leading journals and their impact and influence in the global research community. The two key measures used in this analysis to assess the journals in which the EPA safe pesticides/safe products papers are published are the Impact Factor and Immediacy Index. The Impact Factor is a measure of the frequency with which the “average article” in a journal has been cited in a particular year.  The Impact Factor helps evaluate a journal’s relative importance, especially when compared to other journals in the same field.  The Immediacy Index is a measure of how quickly the “average article” in a journal is cited.  This index indicates how often articles published in a journal are cited within the same year and it is useful in comparing how quickly journals are cited. 

The report includes a summary of the results of the analysis, an analysis of the 357 safe pesticides/safe products papers analyzed by ESI field (e.g., clinical medicine, environment/ecology, plant & animal science), an analysis of the journals in which the safe pesticides/safe products papers were published, a table of the highly cited researchers in the Safe Pesticides/Safe Products Research Program, and a list of the patents and patent applications resulting from the program.

SUMMARY OF RESULTS
  1. Nearly one-quarter of the safe pesticides/safe products publications are highly cited papers.  A review of the citations indicates that 82 (23%) of the safe pesticides/safe products papers qualify as highly cited when using the ESI criteria for the top 10% of highly cited publications.  This is 2.3 times the number expected.  Ten (2.8%) of the safe pesticides/safe products papers qualify as highly cited when using the ESI criteria for the top 1%, which is 2.8 times the number expected.  One (0.28%) of these papers qualifies as very highly cited when using the criteria for the top 0.1%, which is 2.8 times the number anticipated.  As expected, no papers actually meet the 0.01% threshold for the most highly cited papers (the expected number for this size program is 0.04 papers in the most highly cited category).
  2. The safe pesticides/safe products papers are more highly cited than the average paper.  Using the ESI average citation rates for papers published by field as the benchmark, in 12 of the 16 fields in which the EPA safe pesticides/safe products papers were published, the ratio of actual to expected cites is greater than 1, indicating that the safe pesticides/safe products papers are more highly cited than the average papers in those fields. For all 16 fields combined, the ratio of total number of cites to the total number of expected cites (3,534 to 2,583.17) is 1.37, indicating that the safe pesticides/safe products papers are more highly cited than the average paper.
  3. More than one-third of the safe pesticides/safe products papers are published in high impact journals.  One hundred twenty-five (125) of the 357 papers were published in the top 10% of journals ranked by JCR Impact Factor, representing 35% of EPA’s safe pesticides/safe products papers. This number is 3.5 times higher than expected. One hundred eleven (111) of the 357 papers appear in the top 10% of journals ranked by JCR Immediacy Index, representing 31% of EPA’s safe pesticides/safe products papers. This number is 3.1 times higher than expected.
  4. Three of the safe pesticides/safe products papers qualify as hot papers. Using the hot paper thresholds established by ESI as a benchmark, 3 hot papers, representing 1.2% of the safe pesticides/safe products papers, were identified in the analysis. Hot papers are papers that are highly cited shortly after they are published. The number of safe pesticides/safe products hot papers is 12 times higher than expected.
  5. The authors of the safe pesticides/safe products papers cite themselves much less than the average author.  One hundred ninety-four (194) of the 3,534 cites are author self-cites. This 5.5% author self-citation rate is well below the accepted range of 10-30% author self-citation rate.
  6. Eighteen of the authors are included in Thomson’s ISIHighlyCited.com, a database of the world’s most influential researchers who have made key contributions to science and technology from 1981 to 1999.

Highly Cited Safe Pesticides/Safe Products Publications

The 357 safe pesticides/safe products papers reviewed for this analysis covered 16 of the 22 ESI fields.  The distribution of the papers among these 16 fields and the number of citations by field are presented in Table 1.

Table 1. Safe Pesticides/Safe Products Papers by ESI Fields

No. of Citations
ESI Field
No. of EPA SP/SP Papers
Average Cites/Paper
1,462  
Environment/Ecology
139
10.52
563  
Pharmacology & Toxicology
72
7.82
325  
Plant & Animal Science
35
9.28
219  
Chemistry
20
10.95
190  
Microbiology
7
27.14
182   
Molecular Biology & Genetics
19
9.58
180  
Biology & Biochemistry
16
11.25
124  
Neuroscience & Behavior
14
8.86
93  
Multidisciplinary
4
23.25
74  
Clinical Medicine
12
6.17
56  
Engineering
10
5.60
26  
Agricultural Sciences
4
6.50
21  
Immunology
1
21.00
10  
Computer Science
2
5.00
8   
Economics & Business
1
8.00
1  
Geosciences
1
1.00
Total = 3,534
Total =   357
9.90

There are 82 (23.0% of the papers analyzed) highly cited EPA safe pesticides/safe products papers in 14 of the 16 fields—Environment/Ecology, Plant & Animal Science, Pharmacology & Toxicology, Microbiology, Chemistry, Multidisciplinary, Biology & Biochemistry, Neuroscience & Behavior, Clinical Medicine, Engineering, Molecular Biology & Genetics, Economics & Business, Agricultural Sciences, and Computer Science—when using the ESI criteria for the top 10% of papers.  Table 2 shows the number of EPA papers in those 14 fields that meet the top 10% threshold in ESI

Ten (2.8%) of the papers analyzed qualify as highly cited when using the ESI criteria for the top 1% of papers.  These papers cover four fields—Environment/Ecology, Pharmacology & Toxicology, Plant & Animal Science, and Engineering.  Table 3 shows the 10 papers by field that meet the top 1% threshold in ESI.  The citations for these 10 papers are provided in Tables 4 through 7. 

There was 1 (0.28%) very highly cited safe pesticides/safe products paper in the field of Environment/Ecology. This paper, which met the top 0.1% threshold in ESI, is listed in Table 8.  None of the safe pesticides/safe products papers actually met the top 0.01% threshold in ESI, which is to be expected.

Table 2. Number of Highly Cited Safe Pesticides/Safe Products Papers by Field (top 10%)

Citations
ESI Field
No. of Papers
Average Cites/Paper
% of EPA Papers in Field
810
Environment/Ecology
35
23.14
25.18%
261
Plant & Animal Science
16
16.31
45.71%
160
Pharmacology & Toxicology
14
11.43
19.44%
124
Microbiology
2
62.00
28.57%
118
Chemistry
2
59.00
10.00%
93
Multidisciplinary
2
46.50
50.00%
83
Biology & Biochemistry
2
41.50
12.50%
68
Neuroscience & Behavior
1
68.00
7.14%
42
Clinical Medicine
2
21.00
16.67%
34
Engineering
2
17.00
20.00%
27
Molecular Biology & Genetics
1
27.00
5.26%
8
Economics & Business
1
8.00
100.00%
6
Agricultural Sciences
1
6.00
25.00%
3
Computer Science
1
3.00
50.00%
Total = 1,837
Total = 82
 
22.97%

Table 3. Number of Highly Cited Safe Pesticides/Safe Products Papers by Field (top 1%)

Citations
ESI Field
No. of Papers
Average Cites/Paper
% of EPA Papers in Field
121

Environment/Ecology

4

30.25

2.88%

12

Pharmacology & Toxicology

4

3.00

5.56%

9

Plant & Animal Science

1

9.00

2.86%

3

Engineering

1

3.00

10.00%

Total = 145

 

Total = 10

 

2.84%

Table 4. Highly Cited Safe Pesticides/Safe Products Papers in the Field of Environment/Ecology (top 1%)

No. of Cites
First Author
Paper
89

Ankley GT

Description and evaluation of a short-term reproduction test with the fathead minnow (Pimephales promelas).  Environmental Toxicology and Chemistry 2001;20(6):1276-1290.

10

Tietge JE

Metamorphic inhibition of Xenopus laevis by sodium perchlorate: Effects on development and thyroid histology.  Environmental Toxicology and Chemistry 2005;24(4):926-933.

16

Shafer TJ

Developmental neurotoxicity of pyrethroid insecticides: critical review and future research needs. Environmental Health Perspectives 2005;113(2): 123-136.

6

Garrison AW

Probing the enantioselectivity of chiral pesticides.  Environmental Science & Technology 2006;40(1):16-23.

Table 5. Highly Cited Safe Pesticides/Safe Products Papers in the Field of
Pharmacology & Toxicology (top 1%)

No. of Cites
First Author
Paper
3
Barton HA
The acquisition and application of absorption, distribution, metabolism and excretion (ADME) data in agricultural chemical safety assessments.  Critical Reviews in Toxicology 2006;36(1):9-35.
3
Carmichael NG
Agricultural chemical safety assessment: a multisector approach to the modernization of human safety requirements. Critical Reviews in Toxicology 2006;36(1):1-7.
3
Cooper RL
A tiered approach to life stages testing for agricultural chemical safety assessment. Critical Reviews in Toxicology 2006;36(1):69-98.
3
Doe JE
A tiered approach to systemic toxicity testing for agricultural chemical safety assessment. Critical Reviews in Toxicology 2006;36(1):37-68.

Table 6.  Highly Cited Safe Pesticides/Safe Products Papers in the Field of
Plant & Animal Science (top 1%)

No. of Cites
First Author
Paper
9
Greytak SR
Isolation and characterization of two cytochrome P450 aromatase forms in killifish (Fundulus heteroclitus): differential expression in fish from polluted and unpolluted environments.  Aquatic Toxicology 2005;71(4): 371-389.

Table 7.  Highly Cited Safe Pesticides/Safe Products Paper in the Field of Engineering (top 1%)

No. of Cites First Author
Paper
3
Ankley GT
The fathead minnow in aquatic toxicology: past, present and future.  Aquatic Toxicology 2006;78(1):91-102.

Table 8.  Very Highly Cited Safe Pesticides/Safe Products Papers (top 0.1%)

ESI Field
No. of Cites
First Author
Paper
Environment/ Ecology
6
Garrison AW
Probing the enantioselectivity of chiral pesticides.  Environmental Science & Technology 2006;40(1):16-23.

fe Pesticides/Safe Products Papers (top 0.1%)

Ratio of Actual Cites to Expected Citation Rates

The expected citation rate is the average number of cites that a paper published in the same journal in the same year and of the same document type (article, review, editorial, etc.) has received from the year of publication to the present.  Using the ESI average citation rates for papers published by field as the benchmark, in 12 of the 16 fields in which the EPA safe pesticides/safe products papers were published, the ratio of actual to expected cites is greater than 1, indicating that the safe pesticides/safe products papers are more highly cited than the average papers in those fields (see Table 9). For all fields combined, the ratio of total number of cites to total number of expected cites (3,534 to 2,583.17) is 1.37, indicating that the safe pesticides/safe products papers are more highly cited that the average paper.

Table 9.  Ratio of Actual Cites to Expected Cites for
Safe Pesticides/Safe Products Papers by Field

ESI Field
Total Cites
Expected Cite Rate
Ratio
Agricultural Sciences
26
15.09
1.72
Biology & Biochemistry
180
151.87
1.18
Chemistry
219
131.53
1.66
Clinical Medicine
74
41.14
1.80
Computer Science
10
4.33
2.31
Economics & Business
8
1.28
6.25
Engineering
56
27.97
2.00
Environment/Ecology
1,462
854.74
1.71
Geosciences
1
4.70
0.21
Immunology
21
22.11
0.95
Microbiology
190
105.59
1.80
Molecular Biology & Genetics
182
373.19
0.49
Multidisciplinary
93
11.11
8.37
Neuroscience & Behavior
124
135.63
0.91
Pharmacology & Toxicology
563
547.25
1.03
Plant & Animal Science
325
155.64
2.09
All Fields Combined
3,534
2,583.17
1.37

JCR Benchmarks

Impact Factor.  The JCR Impact Factor is a well known metric in citation analysis.  It is a measure of the frequency with which the “average article” in a journal has been cited in a particular year.  The Impact Factor helps evaluate a journal’s relative importance, especially when compared to others in the same field.  The Impact Factor is calculated by dividing the number of citations in the current year to articles published in the 2 previous years by the total number of articles published in the 2 previous years. 

Table 10 indicates the number of safe pesticides/safe products papers published in the top 10% of journals, based on the JCR Impact Factor.  One hundred twenty-five (125) of 357 papers were published in the top 10% of journals, representing 35% of EPA’s safe pesticides/safe products papers. This indicates that more than one-third of the safe pesticides/safe products papers are published in the highest quality journals as determined by the JCR impact factor, which is 3.5 times the expected percentage.

Table 10.  Safe Pesticides/Safe Products Papers in Top 10% of Journals by JCR Impact Factor

EPA SP/SP
Papers in that
Journal
Journal
Impact Factor
(IF)
JCR IF Rank
1
Science
30.927
6
1
Nature
29.273
11
1
JAMA-Journal of the American Medical Association
23.332
19
2
Proceedings of the National Academy of Sciences of the United States of America
10.231
89
1
Molecular & Cellular Proteomics
9.876
99
1
Human Mutation
7.923
145
1
Progress in Nuclear Magnetic Resonance Spectroscopy
6.462
201
1
Journal of Immunology
6.387
205
1
Molecular Biology and Evolution
6.233
211
1
Bioinformatics
6.019
224
1
Analytical Chemistry
5.635
242
24
Environmental Health Perspectives
5.342
257
3
Mutation Research-Reviews in Mutation Research
5.333
259
1
Endocrinology
5.313
261
2
American Journal of Epidemiology
5.068
290
4
Critical Reviews in Toxicology
5.000
296
1
Physiological Genomics
4.636
346
1
Journal of Applied Ecology
4.594
351
1
Ecology
4.506
366
1
Cancer Epidemiology Biomarkers & Prevention
4.460
378
1
Biotechnology Advances
4.455
381
1
Biochimica et Biophysica Acta-Bioenergetics
4.302
413
3
Molecular Ecology
4.301
414
1
New Phytologist
4.285
417
1
Conservation Biology
4.110
455
1
Global Change Biology
4.075
464
19
Environmental Science & Technology
4.054
467
1
Drug Metabolism and Disposition
4.015
481
3
Applied and Environmental Microbiology
3.818
544
2
Ecological Applications
3.804
548
1
Mutation Research-Fundamental and Molecular Mechanisms of Mutagenesis
3.340
696
2
Chemical Research in Toxicology
3.339
699
1
Green Chemistry
3.255
722
1
Lung Cancer
3.172
753
4
Toxicology and Applied Pharmacology
3.148
764
2
Journal of Chromatography A
3.096
778
29
Toxicological Sciences
3.088
780
1
Rapid Communications in Mass Spectrometry
3.087
781
1
Water Research
3.019
809
Total = 125

Immediacy Index.  The JCR Immediacy Index is a measure of how quickly the average article in a journal is cited.  It indicates how often articles published in a journal are cited within the year they are published.  The Immediacy Index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year.

Table 11 indicates the number of safe pesticides/safe products papers published in the top 10% of journals, based on the JCR Immediacy Index.  One hundred eleven (111) of the 357 papers appear in the top 10% of journals, representing 31% of the safe pesticides/safe products papers. This indicates that nearly one-third of the safe pesticides/safe products papers are published in the highest quality journals as determined by the JCR immediacy index, which is 3.1 times higher than the expected percentage.

Table 11.  Safe Pesticides/Safe Products Papers in Top 10% of Journals by
JCR Immediacy Index

EPA SP/SP
Papers in that
Journal
Journal
Immediacy Index
(II)
JCR II Rank
1
Science
6.398
6
1
Nature
5.825
11
1
JAMA-Journal of the American Medical Association
5.082
17
1
Molecular Biology and Evolution
1.832
109
2
Proceedings of the National Academy of Sciences of the United States of America
1.746
121
1
ILAR Journal
1.595
137
1
Human Mutation
1.395
172
1
Molecular & Cellular Proteomics
1.377
177
2
International Journal of Toxicology
1.309
193
1
Endocrinology
1.260
210
3
Mutation Research-Reviews in Mutation Research
1.143
252
1
New Phytologist
1.125
258
1
Progress in Nuclear Magnetic Resonance Spectroscopy
1.111
266
2
American Journal of Epidemiology
1.099
271
1
Physiological Genomics
1.058
287
24
Environmental Health Perspectives
0.955
346
1
Bioinformatics
0.944
354
1
Journal of Immunology
0.906
381
1
Biochimica et Biophysica Acta-Bioenergetics
0.895
390
3
Ecotoxicology
0.846
434
1
Lung Cancer
0.735
532
1
Drug Metabolism and Disposition
0.733
534
3
Science of the Total Environment
0.731
538
2
Chemical Research in Toxicology
0.729
542
1
Journal of Applied Ecology
0.726
551
1
Analytical Chemistry
0.713
569
1
Journal of Experimental Biology
0.684
601
1
Mutation Research-Fundamental and Molecular Mechanisms of Mutagenesis
0.682
604
1
Green Chemistry
0.631
695
8
Human and Ecological Risk Assessment
0.628
699
1
Ecology
0.621
710
29
Toxicological Sciences
0.617
716
3
Molecular Ecology
0.598
742
1
Biological Conservation
0.589
762
1
Cancer Epidemiology Biomarkers & Prevention
0.579
785
1
Bioorganic & Medicinal Chemistry
0.577
791
3
Journal of Exposure Analysis and Environmental Epidemiology
0.571
801
2
Ecological Applications
0.543
870
Total = 111

Hot Papers

ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorter—papers must be cited within 2 years of publication and the citations must occur in a 2-month time period.  Papers are assigned to 2-month periods and thresholds are set for each period and field to select 0.1% of papers.  There were no hot papers identified for the current 2-month period (i.e., July-August 2006), but there were a number of hot papers identified from previous periods.

Using the hot paper thresholds established by ESI as a benchmark, 8 hot papers, representing 2.2% of the safe pesticides/safe products papers, were identified in two fields—Environment/Ecology and Pharmacology & Toxicology.  The hot papers are listed in Table 12.

Table 12.  Hot Papers Identified Using ESI Thresholds

Field
ESI
Hot Papers
Threshold
No. of Cites
in 2-Month
Period
Paper
Environment/ Ecology
3
4 cites in April-May 2004
Schumaker NH, et al.  Projecting wildlife responses to alternative future landscapes in Oregon’s Willamette Basin.  Ecological Applications 2004;14(2):381-400.
Environment/ Ecology
3
3 cites in April-May 2004
Baker JP, et al.  Alternative futures for the Willamette River Basin, Oregon.  Ecological Applications 2004;14(2):313-324.
3
3 cites in November-December 2003
Schreinemachers DM.  Birth malformations and other adverse perinatal outcomes in four US wheat-producing states.  Environmental Health Perspectives 2003;111(9): 1259-1264.
3
3 cites in March-April 2001
Cory-Slechta DA, et al.  Methods to identify and characterize developmental neurotoxicity for human health risk assessment. I: Behavioral effects.  Environmental Health Perspectives 2001;109(Suppl 1):79-91.
Pharmacology & Toxicology
2
3 cites in January-February  2006
Carmichael NG, et al.  Agricultural chemical safety assessment: A multisector approach to the modernization of human safety requirements. Critical Reviews in Toxicology 2006;36(1):1-7.
2
3 cites in January-February  2006
Cooper RL, et al.  A tiered approach to life stages testing for agricultural chemical safety assessment.  Critical Reviews in Toxicology 2006;36(1):69-98.
2
3 cites in January-February  2006
Doe JE, et al.  A tiered approach to systemic toxicity testing for agricultural chemical safety assessment.  Critical Reviews in Toxicology 2006;36(1):37-68.
2
2 cites in January-February  2006
Barton HA, et al.  The acquisition and application of absorption, distribution, metabolism and excretion (ADME) data in agricultural chemical safety assessments. Critical Reviews in Toxicology 2006;36(1):9-35.

Author Self-Citation

Self-citations are journal article references to articles from that same author (i.e., the first author).  Because higher author self-citation rates can inflate the number of citations, the author self-citation rate was calculated for the safe pesticides/safe products papers.  Of the 3,534 total cites, 194 are author self-cites—a 5.5% author self-citation rate.  Garfield and Sher 3 found that authors working in research-based disciplines tend to cite themselves on the average of 20% of the time.  MacRoberts and MacRoberts 4 claim that approximately 10% to 30% of all the citations listed fall into the category of author self-citation. Kovacic and Misak 5 recently reported a 20% author self-citation rate for medical literature.  Therefore, the 5.5% self-cite rate for the safe pesticides/safe products papers is well below the range for author self-citation.

Highly Cited Researchers

A search of Thomson’s ISIHighlyCited.com revealed that 18 (1.7%) of the 1,062 authors of the safe pesticides/safe products papers are highly cited researchers.  ISIHighlyCited.com is a database of the world’s most influential researchers who have made key contributions to science and technology during the period from 1981 to 1999. The highly cited researchers identified during this analysis of the safe pesticides/safe products publications are presented in Table 13.

Table 13.  Highly Cited Researchers Authoring Safe Pesticides/Safe Products Publications

Highly Cited Researcher
Affiliation
ESI Field
Ankley, Gerald T
U.S. EPA
Environment/Ecology
Bollag, Jean-Marc
Pennsylvania State University
Environment/Ecology
Boobis, Alan R.
Imperial College London
Pharmacology
Fuchs, Elaine
The Rockefeller University
Molecular Biology & Genetics
Giesy, John P.
University of Saskatchewan
Environment/Ecology
Gray, Jr., Leon Earl
U.S. EPA
Pharmacology
Kadlubar, Fred F.
U.S. Food and Drug Administration
Pharmacology
Kimber, Ian
Syngenta Central Toxicology Laboratory
Pharmacology
Melillo, Jerry M.
Marine Biological Laboratory, Woods Hole
Environment/Ecology
Muir, Derek C.G.
Environment Canada
Environment/Ecology
Engineering
Needham, Larry L.
Centers for Disease Control and Prevention
Environment/Ecology
Roelofs, Wendell L. 
Cornell University
Environment/Ecology
Rubin, Gerald M.
University of California-Berkeley
Molecular Biology & Genetics
Smith, Gary C.
Colorado State University
Agricultural Sciences
Springer, Timothy A.
Wildlife International, Ltd.
Molecular Biology & Genetics
Immunology
Clinical Medicine
Suidan, Makram T.
University of Cincinnati
Environment/Ecology
Sumpter, John P.
Brunel University
Environment/Ecology
Yang, Chung S.
Rutgers, The State University of New Jersey
Pharmacology
Total = 18
 

Patents

There were no patents or patent applications associated with this research program.


1     Thomson Scientific’s Web of Science provides access to current and retrospective multidisciplinary information from approximately 8,830 of the most prestigious, high impact research journals in the world. Web of Science also provides cited reference searching.

2     Scopus is a large abstract and citation database of research literature and quality Web sources designed to support the literature research process. Scopus offers access to 15,000 titles from 4,000 different publishers, more than 12,850 academic journals (including coverage of 535 Open Access journals, 750 conference proceedings, and 600 trade publications), 27 million abstracts, 245 million references, 200 million scientific Web pages, and 13 million patent records. 


3     Garfield E, Sher IH.  New factors in the evaluation of scientific literature through citation indexing.  American Documentation 1963;18(July):195-210.

4     MacRoberts MH, MacRoberts BR.  Problems of citation analysis: a critical review.  Journal of the American Society of Information Science 1989;40(5):342-349.

5     Kavaci N, Misak A.  Author self-citation in medical literature.  Canadian Medical Association Journal 2004;170(13):1929-1930.

 

Top of page


Local Navigation


Jump to main content.