The Impact of Record Scattering on the Measurement of Immunization Coverage
Shannon Stokley, MPH*; Lance E. Rodewald, MD*; and Edmond F. Maes, PhD‡

ABSTRACT. Background. Lack of a consolidated im- munization record may lead to problems with determin- ing individual immunization needs at office visits as well as measuring vaccination coverage levels of a clini- cian’s practice or a community’s population.
Objectives. For children with multiple immunization providers, evaluate the difference in coverage levels us- ing data from all responding immunization providers compared with: 1) the most recent immunization provid- er’s records, 2) the first immunization provider’s records, and 3) a randomly selected immunization provider’s records. Identify characteristics of the most recent pro- vider that may be associated with reporting incomplete immunization histories.
Methods. Data from the 1995 National Immunization Provider Record Check Study (NIPRCS) were used for analysis. The NIPRCS is a provider validation study of the household reported immunization histories of all children 19 to 35 months of age included in the National Health Interview Survey (NHIS). Providers identified by the child’s parent during the NHIS interview are mailed a 2-page survey to report all immunizations (type and date) the child received, regardless of the provider who administered the shots, and child’s first and most recent visit dates to the practice.
Results. Of the 1352 children with provider data, 304 (22%) had received immunizations from more than one provider. Compared with information from all providers and depending on the vaccine, the most recent provider records underestimated coverage by 9.6 to 13.4 percent- age points; the initial provider records underestimated coverage by 15.6 to 34.6 percentage points; and the ran- domly selected provider records underestimated cover- age by 10.0 to 20.7 percentage points. Public facilities and having an immunization summary sheet in the patient’s chart were associated with having complete records.
Conclusion. Scattered immunization records signifi- cantly compromise the ability of clinicians to determine the immunization status of their patients who received immunizations at other sites of health care. Routinely assessing immunization coverage levels at the practice level, implementing a recall system, and developing community-wide immunization registries are some strat- egies to reduce the problem of scattered immunization records. Pediatrics 2001;107:91–96; immunization, assess- ment, provider validation, record scattering.
ABBREVIATIONS. UTD, up-to-date; NIPRCS, National Immuni- zation Provider Record Check Study; NHIS, National Health In- terview Survey; DTP, diphtheria–tetanus–pertussis; MMR, mea- sles–mumps–rubella; HIB, Haemophilus influenzae type b; PPV, positive predictive value; NPV, negative predictive value.

Routine childhood vaccination is one of the ma- jor public health success stories in this centu- ry,^1 currently producing the lowest incidence rates of traditional vaccine-preventable diseases^2 at- tributed to the highest vaccination coverage levels for the corresponding diseases.^3,4 However, the con- tinued success of vaccination is being challenged by an immunization schedule that is increasing in size and complexity and by the lack of accessible, unified immunization records. Already, children are recom- mended to receive a total of 15 to 19 vaccine doses by 18 months of age, compared with only 8 doses by 20 years ago. New vaccines will continue and perhaps accelerate this trend.
A child usually has 2 sources of immunization history information, the parent and the health care provider. Because parental records of their child’s immunization history have been shown to be unre- liable^5–8 and because health care professionals are required by law to record information about immu- nizations given in their offices, the health care pro- vider’s records are typically viewed as being the most accurate and reliable. However, recent studies have shown that many children are vaccinated away from the primary care office,^9,10 either at a previous provider office or at the health department; most new patients do not bring adequate documentation of their immunization history to the initial visit to a new primary care provider^8,11; and communication among immunization providers is frequently poor,^12 all of which leads to a lack of unified records at the primary care provider office and an inability to de- termine vaccination needs accurately.
Lack of a consolidated record is problematic not only for determining individual immunization needs at office visits, but also for measuring the vaccination coverage levels of a clinician’s practice or a commu- nity’s population. Measuring coverage levels at the practice or community level is an important strategy to improve and sustain high vaccination coverage levels.^13,14 In theory, the relation between missing vaccinations and misclassification of an up-to-date (UTD) child as not UTD is exponential, with small amounts of missing information having a very large impact on the accuracy of coverage assessments.^15
From the *National Immunization Program and the ‡Epidemiology Pro- gram Office, Centers for Disease Control and Prevention, Atlanta, Georgia. Presented in part at the Pediatric Academic Societies Annual Meeting; May 1, 1999; San Francisco, CA.
Received for publication Sep 2, 1999; accepted Mar 31, 2000. Reprint requests to (S.S.) Centers for Disease Control and Prevention/ National Immunization Program, 1600 Clifton Rd, E-62, Atlanta, GA 30333. E-mail: sstokley@cdc.gov
PEDIATRICS (ISSN 0031 4005). Copyright © 2001 by the American Acad- emy of Pediatrics.

PEDIATRICS Vol. 107 No. 1 January 2001 91

Scattered records are a potential source of missing vaccination information at the provider and commu- nity level.
We report the results of a national survey to mea- sure the impact that the scattering of vaccination records has on the assessment of coverage for US preschool children. We use the 1995 National Immu- nization Provider Record Check Study (NIPRCS) to:
1) determine the proportion of children having more than one immunization provider and 2) compare coverage assessments using records available at in- dividual providers’ offices with a consolidated im- munization record.

METHODS
Data from the 1995 NIPRCS, a component of the National Health Interview Survey (NHIS), were analyzed. The NHIS is a cross-sectional, household, face-to-face interview of the civilian, noninstitutionalized population of the United States. Beginning in 1991, the NHIS included immunization-related questions to pro- duce national vaccination coverage estimates of children ,6 years of age.^16 During the immunization section of the NHIS, the re- spondent could choose to report from the child’s immunization record all immunizations (type and date) the child received or the respondent could report from memory the number of doses (all is an acceptable response) of each immunization the child had re- ceived.
The NIPRCS was introduced in 1994 to reduce response bias of reporting immunizations from memory to the NHIS.^17 For each data collection year, immunization histories of children 19 to 35 months of age collected during the NHIS interview were verified by the provider(s) reported by the parent to have administered the child’s vaccines. An immunization history questionnaire was mailed to each identified provider. The first page of the question- naire included an immunization grid for the provider to record the date and type of each immunization that the child received and to indicate whether the vaccine was given by a different provider. The second page included questions about the child’s first and most recent visit dates to the practice, the source of immunization information from the patient’s chart used to complete the ques- tionnaire (ie, immunization summary sheet, physician’s orders from patient’s chart, or an immunization summary sheet plus an additional source), and characteristics of the practice (public or private facility).
For children with more than one provider, we compared the most recent visit dates of each provider to identify the most recent provider for each child in the survey. We compared the first visit dates of each provider to identify the first provider. If the infor- mation was missing for either the first or most recent visit date, we used the information in the immunization grid to identify the provider sequence. The date of the earliest immunization given by the provider (the vaccine was not indicated to be given elsewhere) was recorded as the first visit date and the most recent immuni- zation date reported by the provider (even if the provider indi- cated that the vaccine was administered elsewhere) was recorded as the most recent visit date.
We determined the immunization coverage level for each vac- cine and vaccine series using: 1) data from all provider records (referred to as the consolidated record), 2) data from only the most recent provider’s records, 3) data from only the first provider’s records, and 4) data from only one randomly selected provider’s records. To determine the immunization coverage levels based on one randomly selected provider, a bootstrap simulation of 1000 samples of one randomly selected provider per child was per- formed. The immunization coverage levels from the randomly selected provider’s records represent the average immunization coverage levels taken from the 1000 samples. Differences in cov- erage levels were calculated by comparing data from the most recent provider, the first provider, or a randomly selected pro- vider to data from the consolidated record. The vaccines and combination of vaccines evaluated include: 4 or more doses of diphtheria–tetanus–pertussis (DTP), 3 or more doses of polio, 1 or more doses of measles–mumps–rubella (MMR), 3 or more doses of Haemophilus influenzae type b (HIB), 3 or more doses of hepatitis B, the 4:3:1 series (4-DTP/3-polio/1-MMR), the 4:3:1:3 series (4- DTP/3-polio/1-MMR/3-HIB), and the 4:3:1:3:3 series (4-DTP/3- polio/1-MMR/3-HIB/3-hepatitis B). Varicella was not licensed at the beginning of the 1995 NHIS data collection and was not included in our study.
Sensitivity, specificity, and predictive value analysis of the most recent provider was performed to determine whether the ability to identify undervaccinated children differed by provider characteristics. The characteristics of interest were facility type and the source of information used to complete the immunization history questionnaire. The analysis was limited to children who received immunizations from more than one provider and the true immunization status was determined from the overall immuniza- tion history resulting from the consolidated record. The condition being tested was the recent provider’s ability to accurately identify a child as not UTD for the recommended immunizations as de- scribed previously.
The specificity of the most recent provider’s records represents the ability to correctly identify children who were truly UTD and, therefore, not in need of additional vaccination. We defined spec- ificity as the proportion of children UTD according to the consol- idated record who were also identified as UTD based on the most recent provider’s records. Low specificity implies that children not in need of vaccination would be incorrectly classified as needing vaccination, which might result in overvaccination.
The positive predictive value (PPV) of the most recent provid- er’s records represents the probability that a child classified as not UTD was truly not UTD. We define PPV as the proportion of children not UTD based on the most recent provider’s records, who were truly not UTD based on the consolidated record.
Because the most recent provider’s records are a subset of the consolidated record, by definition, children not UTD according to the consolidated record cannot be UTD according to the most recent provider’s records. Therefore, the sensitivity and negative predictive value (NPV) to identify underimmunized children will always be 100%. Thus, we do not report sensitivity or NPV as results.
RESULTS
Of the 2089 children 19 to 35 months of age in- cluded in the 1995 NHIS, 1352 (65%) had provider immunization data, of which 304 (22%) went to more than one provider (91% went to 2 providers, 8% went to 3 providers, and 1% went to 4 providers). Overall, there were 769 children (57%) with provider data who had immunization dates reported from a writ- ten record during the NHIS interview, whereas 202 children (66%) who went to more than one provider had immunization dates reported from a written record during the NHIS interview.
Differences in coverage levels for each vaccine and vaccine series for children with more than one pro- vider can be seen in Table 1. Compared with infor- mation from the consolidated record, the most recent provider records underestimated coverage by 9.8 percentage points for 3 or more doses of hepatitis B to 12.3 percentage points for 4 or more doses of DTP. The first providers’ records underestimated coverage by 27.4 to 45.8 percentage points and the randomly selected providers underestimated coverage by 18.7 to 29.1 percentage points, depending on the vaccine or vaccine series. Vaccine series that require a vac- cine dose in the second year of life (eg, DTP) had larger differences in coverage levels between the con- solidated record and the first, most recent, and ran- domly selected provider, compared with vaccine se- ries that do not require a vaccine dose in the second year of life (eg, HIB).
To determine how record scattering could affect immunization coverage levels of all children in-

92 IMPACT OF RECORD SCATTERING ON THE MEASUREMENT OF IMMUNIZATION COVERAGE

TABLE 1. Comparison of Difference in Measured Immunization Coverage Levels by Using Immunization Histories From the Most Recent Provider, the First Provider, and One Randomly Selected Provider With Histories Combined Across All Providers, Children With More Than One Provider (n 5 304), 1995 NIPRCS
Vaccines Coverage Level
Using Consolidated Record Difference in Coverage Level Using Only
Most Recent Provider Records Difference in Coverage Level Using Only First Provider Records
Difference in Coverage Level Using Only One Randomly Selected
Provider Records

41 DTP/DTaP 78.2 212.3 245.8 228.9 31 Polio 86.2 213.4 244.3 227.7 11 MMR 90.5 211.9 244.3 227.7 31 HIB 90.5 211.3 227.4 219.5 31 Hepatitis B 66.9 29.8 227.5 218.7 4:3:1* 76.6 212.8 245.0 228.9 4:3:1:3† 75.2 212.9 245.1 229.1 4:3:1:3:3‡ 54.1 212.4 231.1 222.1
* 4:3:1 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, and 11 doses of MMR. † 4:3:1:3 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, 11 doses of MMR, and 31 doses of HIB. ‡ 4:3:1:3:3 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, 11 doses of MMR, 31 doses of HIB, and 31 doses of hepatitis B.
ARTICLES 93
cluded in the NIPRCS, differences in coverage levels based on the most recent providers records com- pared with the consolidated record were calculated for all children with provider data regardless of the number of providers visited. For children with only one provider, that provider was considered the most recent provider. Compared with the consolidated record, the most recent provider underestimated by
3.5 percentage points for 3 or more doses of HIB, 4.2 percentage points for 4 or more doses of DTP/DTaP, and 4.5 percentage points for the 4:3:1:3:3 series.
Figure 1 presents a closer look at how accurately the most recent provider classified a child as UTD for a particular vaccine or series. Of the children with more than one provider who were UTD for a partic- ular vaccine or series based on information from the consolidated record, the most recent provider classi- fied 15.8% of the children as behind schedule for 41 DTP/DTaP and 22.9% of the children as behind schedule for the 4:3:1:3:3 series.
Of the children who went to more than one pro- vider, 64% of the recent providers practiced in a private facility, 23% in a public facility, 6% were “other” (hospital outpatient, military, other), and 6% were unknown. Because a small proportion of facil- ities were other or unknown, analysis by facility type was limited to private and public facilities. Table 2 shows the specificity and predictive value analysis of the most recent provider by facility type for children who went to more than one provider. Differences in specificity were found between public and private providers. Although public facilities had a higher probability of correctly classifying a child who was UTD for all vaccine and vaccine series compared with private facilities, a significant difference was found only for the 4:3:1 series. In addition, significant differences in PPV were found between private and public facilities for 41 DTP/DTaP, 31 polio, the 4:3:1 series, and the 4:3:1:3:3 series. If a public facility classified a child as not UTD for the 4:3:1 series, there was a 90% chance that the child was truly not UTD, whereas if a private facility classified a child as not UTD, there was only a 58% chance that the child was not actually UTD.
With respect to the source of information the most recent provider used to complete the immunization history questionnaire, 60% used an immunization summary sheet, 10% used physician’s orders as doc- umented in the patient’s chart, 22% used a combina- tion of a summary sheet and another source, 5% used an other source, and 3% did not specify. Because of the small sample size, specificity and PPV analysis (Table 3) excluded the most recent providers who used an other source or did not specify. Comparison of the most recent providers who used physician’s orders from the patient’s chart to the recent provid- ers who used the immunization summary sheet shows significant differences in specificity for all vac-

Fig 1. Proportion of UTD children classi- fied as not UTD by the most recent pro- vider, children with more than one pro- vider (n 5 304), 1995 NIPRCS.

TABLE 2. Specificity* and PPV† of Most Recent Provider Identifying Not UTD Children by Facility Type, Children With More Than One Provider, 1995 NIPRCS
Vaccines Private Facility Public Facility

Specificity (%)
PPV
(%)
PPV
(%)
41 DTP/DTaP 82 58 92 86‡ 31 Polio 83 48 92 75‡ 11 MMR 85 38 89 62 31 HIB 88 46 88 55 31 Hepatitis B 85 77 91 86 4:3:1§ 81 58 94‡ 90‡ 4:3:1:3\ 81 62 87 81 4:3:1:3:3¶ 75 76 90 93‡
* Specificity of the most recent provider is the proportion of chil- dren UTD according to all providers who were identified as such by the most recent provider.
† PPV is the probability that the child is truly not UTD given that the most recent provider classified the child as not UTD.
‡ Denotes significantly different (P , .05) from private facility. § 4:3:1 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, and 11 doses of MMR.
\ 4:3:1:3 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, 11 doses of MMR, and 31 doses of HIB.
¶ 4:3:1:3:3 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, 11 doses of MMR, 31 doses of HIB, and 31 doses of hepatitis B.
TABLE 3. Specificity* and PPV† of Most Recent Provider Identifying Not UTD Children by Source of Immunization Information, Children With More Than One Provider, 1995 NIPRCS
Vaccines Immunization
Summary Sheet
Physician’s Orders From Patient’s Chart
Immunization Summary Sheet 1 Other Source

Specificity (%)
PPV
(%)
PPV
(%)
Specificity (%)

cines and vaccine series, except for 31 hepatitis B. Significant differences in PPV were also found be- tween the 2 groups, except for 31 HIB, 31 hepatitis B, and the 4:3:1:3:3 series. No significant differences in specificity and PPV were found between provid- ers who used the immunization summary sheet plus another source and providers who used the immu- nization summary sheet only except for 11 MMR.
Analysis of the source of information used to com- plete the immunization history questionnaire by fa- cility type indicates that for public facilities, 68.5% used an immunization summary sheet only, 14.2% used a combination of an immunization summary sheet and another source, and 3.4% used physician’s orders. Of the private facilities, 60% used an immu- nization summary sheet only, 25.4% used a combi- nation of an immunization summary sheet and an- other source, and 10.8% used physician’s orders.
DISCUSSION
We have shown that scattered immunization records significantly compromise the ability of clini- cians to determine the immunization status of their patients who received immunizations at other sites of health care. Nationally, 22% of children received their early preschool vaccinations from more than one health care professional. Among children having more than one immunization provider: 1) the records of the child’s most recent provider mistakenly indi- cated that 23% of completely vaccinated children were in need of vaccination, 2) a record from the most recent provider indicating that the child needed additional vaccinations for the 4:3:1:3 series was in- correct 38% of the time for private practitioners and 19% of the time for health department clinics, and 3) the presence of a summary immunization record in the chart was associated with more complete records.

Relation to Other Literature
Findings from our study provides a national per- spective to a problem that has been studied mostly at the local level. Hamlin and colleagues^18 showed that records were scattered between 2 clinics located to- gether in Los Angeles County. One clinic was a health department clinic and the other was a com- munity health center; both were on the same floor of the same building. Murphy and colleagues^19 demon- strated that for children who visited a Dallas county public clinic, incomplete documentation of immuni- zations in both the public clinic and parent’s record was associated with a 45% rate of unnecessary im- munization. Yawn and colleagues^20 demonstrated the high degree of record scattering in Olmstead

PPV
(%)
41 DTP/DTaP 87 70 61‡ 34‡ 81 55 31 Polio 89 61 59‡ 14‡ 84 49 11 MMR 90 55 66‡ 8‡ 84 20‡ 31 HIB 91 55 72‡ 24 87 39 31 Hepatitis B 89 76 76 81 79 77 4:3:1§ 87 73 61‡ 34‡ 81 55 4:3:1:3\ 86 72 61‡ 34‡ 81 55 4:3:1:3:3¶ 81 81 52‡ 75 70 74
* Specificity of the most recent provider is the proportion of children UTD according to all providers who were identified as such by the most recent provider.
† PPV is the probability that the child is truly not UTD given that the most recent provider classified the child as not UTD.
‡ Denotes significantly different (P , .05) from private facility. § 4:3:1 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, and 11 doses of MMR. \ 4:3:1:3 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, 11 doses of MMR, and 31 doses of HIB.
¶ 4:3:1:3:3 is defined as having received 41 doses of DTP/DTaP, 31 doses of polio, 11 doses of MMR, 31 doses of HIB, and 31 doses of hepatitis B.

94 IMPACT OF RECORD SCATTERING ON THE MEASUREMENT OF IMMUNIZATION COVERAGE
Specificity (%)
Specificity (%)

County, Minnesota, and showed that if the records could be combined by an immunization registry, the ability to correctly classify children’s immunization status would increase. Our study shows that scat- tered records is a problem that is national in scope, resulting in the misclassification of over 150 000 com- pletely vaccinated US children as being in need of vaccination and that the problem is more serious among private practitioners.
Watson and colleagues^11 showed one source for scattered records—only 22% of parents brought their immunization records to an initial visit to a new primary care clinician. Many studies have shown that parents do not know the immunization status of their children. In general, parents believe that their children are fully vaccinated when they may not be UTD.^5–8,11 Thus, parents cannot be relied on to know the vaccination needs of their children. Our study did not investigate parent recall or the causes for scattered records. Thus, these other studies provide potential explanations for our findings.

Strengths and Weaknesses
The main strengths of the study are that it is na- tional in scope, uses a household area probability survey to identify children, and uses a parent-linked provider record check to determine immunization status. There are 2 main weaknesses of the study. The first is that there is an unknown degree of misclassi- fication of whether the immunization provider was the first, second, or most recent provider. The study methods used the first and last dates of visits to each provider to make that determination, and in 19% of cases, that information was missing. However, the majority of children had only 2 providers, therefore, increasing the degree of certainty when identifying the most recent provider. The second weakness is one of scope. We were limited to the questions avail- able on the NHIS and its immunization provider record check study. Thus, reasons for the scattered records could not be determined.

Implications of the Study
There are several reasons for needing complete immunization records at the offices of immunization providers. As a result of the studies that demon- strated the potential to improve immunization cov- erage by reducing missed opportunities,^21,22 provid- ers are strongly encouraged to vaccinate at every opportunity. For individual children, the records are essential to determine their need for vaccination at the time of the office visit. Failure to assess accurately implies failure to make a correct clinical decision whether to vaccinate and with what vaccines.
For health care providers, complete records are needed to assess accurately the immunization cover- age of their patients—something that all providers, public and private, are being asked to do.^13 Without complete records, the assessments show substan- tially lower coverage than may actually be the case, and information that would help clinicians improve their immunization practices might be rendered in- accurate. However, a benefit of these assessments is to quantify the degree of missing records for clini- cians, which, in turn, should lead to more complete records.
This study also supports previous recommenda- tions that all immunization providers operate a recall system to bring children in need of vaccination back to the office for vaccination and other clinical pre- ventive services.^14,23–25 Recall systems list patients belonging to a practice that cannot be documented to be UTD on immunizations. To avoid recalling fully vaccinated children whose medical records are miss- ing vaccinations that were actually administered, the office staff needs to make a judgment about the completeness of the record. If the staff determines that the record may be incomplete (for example, if the patient is relatively new to the practice or has been recently referred to the health department be- cause his or her insurance does not cover vaccina- tions), an attempt to determine the true immuniza- tion history could precede the recall of the patient.
The final implication of the study is that the data support the development of community immuniza- tion registries that communicate with registries of other communities.^26 Once a system of registries is in place, the problem of scattered immunization records could be greatly reduced in magnitude.

Unanswered Questions and Further Research
This research raises 3 types of questions: What are the causes of scattered records?; What are the conse- quences of scattered records?; and What can be done to reduce their impact? Causation might best be an- swered by looking at the relation between mobility, changing providers, and insurance-related referrals to health departments. Consequences might include both undervaccination and overvaccination. For ex- ample, a reluctance to vaccinate when there is uncer- tainty about the completeness of vaccination records might lead to missed opportunities to vaccinate. Conversely, vaccinating children with incomplete records may lead to overvaccination, as was seen in the study by Murphy and colleagues^19 and Feikema and colleagues.^27 Just as undervaccination exposes children to unnecessary risk of vaccine-preventable diseases, overvaccination exposes children to unnec- essary risk of adverse events from vaccines. Finally, research to reduce the scattering of records will lead to improved efficiency of the immunization delivery system. This work will support and guide the devel- opment of immunization registries, which will ulti- mately help clinicians keep track of the immuniza- tion status and needs of their patients.

REFERENCES
1. Centers for Disease Control and Prevention. Impact of vaccines univer- sally recommended for children—United States, 1900–1998. MMWR Morb Mortal Wkly Rep. 1999;48:243–248
2. Orenstein WA, Hinman AR, Rodewald LE. Public health consider- ations—United States. In: Plotkin SA, Orenstein WA, eds. Vaccines. 3rd ed. Philadelphia, PA: WB Saunders Co; 1999:1024
3. Centers for Disease Control and Prevention. Notice to readers: national vaccination coverage levels among children aged 19–35 months— United States, 1998. MMWR Morb Mortal Wkly Rep. 1999;48:829–830
4. National trends in vaccination coverage. Available at: http:/ www.cdc.gov/nip/coverage/data.htm
5. Suarez L, Simpson DM, Smith DR. Errors and correlates in parental recall of child immunizations: effects on vaccination coverage estimates.
ARTICLES 95

Pediatrics. 1997;99(5). URL: http://www.pediatrics.org/cgi/content/ full/99/5/e3
6. Bolton B, Holt E, Ross A, Hughhart N, Guyer B. Estimating vaccination coverage using parental recall, vaccination cards, and medical records. Public Health Rep. 1998;113:521–526
7. Zell E, Peak R, Rodewald L, Ezzati-Rice T. Vaccine coverage. Public Health Rep. 1999;114:3–4
8. Goldstein KP, Kviz FJ, Daum RS. Accuracy of immunization histories provided by adults accompanying preschool children to a pediatric emergency department. JAMA. 1993;270:2190–2194
9. Zimmerman R, Medsgar A, Ricci E, et al. Impact of free vaccine and insurance status on physician referral of children to public vaccine clinics. JAMA. 1997;278:996–1000
10. Bordley WC, Freed GL, Garrett JM, et al. Factors responsible for immu- nization referrals to health departments in North Carolina. Pediatrics. 1994;94:376–380
11. Watson MA, Feldman KW, Sugar NF, Sommer CJ, Thomas ER, Lin T. Inadequate history as a barrier to immunizations. Arch Pediatr Adolesc Med. 1996;150:135–139
12. Santoli JM, Barker LE, Gandhi NB, Lyons BH, Rodewald L. Health department clinics as immunization providers for pediatric patients in urban and rural settings—a 1998 national survey. Presented at the 1999 Pediatric Academic Societies Annual Meeting; May 1–4, 1999; San Fran- cisco, CA. Abstact No. 764
13. Ad Hoc Working Group for the Development of Standards for Pediatric Immunization Practices. Standards for pediatric immunization prac- tices. JAMA. 1993;269:1817–1822
14. Shefer A, Briss P, Rodewald L, et al. Improving immunization coverage rates: an evidence-based review of the literature. Epi Rev. 1999;21: 96–142
15. Rodewald L, Maes E, Stevenson J, Lyons B, Stokley S, Szilagyi P. Immunization performance measurement in a changing immunization environment. Pediatrics. 1999;103:889–897
16. Massey JT, Moore TF, Parsons VL, et al. Design and Estimation for the National Health Interview Survey, 1985–1994. Hyattsville, MD: US Depart- ment of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention; 1989. Vital and Health Statistics Series 2, No. 110
17. Peak RR, Cadell DM. Overview of the national immunization provider record check study: proceedings of the section on survey research methods. Am Stat Assoc. 1996:332–334
18. Hamlin JS, Wood D, Pereyra M, Grabowsky M. Inappropriately timed immunizations: types, causes, and their relationship to record keeping. Am J Public Health. 1996;86:1812–1814
19. Murphy TV, Pastor P, Medley FB. Factors associated with unnecessary immunization given to children. Pediatr Infect Dis J. 1997;16:47–52
20. Yawn BP, Edmonson L, Huber L, Poland GA, Jacobson RM, Jacobsen SJ. The impact of a simulated immunization registry on perceived child- hood immunization status. Am J Manage Care. 1998;4:185–192
21. Dietz VJ, Stevenson J, Zell ER, Cochi S, Hadler S, Eddins D. Potential impact on vaccination coverage levels by administering vaccines simul- taneously and reducing dropout rates. Arch Pediatr Adolesc Med. 1994; 148:943–948
22. Szilagyi PG, Rodewald LE, Humiston SG, et al. Reducing missed op- portunities for immunizations. Arch Pediatr Adolesc Med. 1996;150: 1193–1200
23. Udovic S, Lieu T. Evidence on office-based interventions to improve childhood immunization delivery. Pediatr Ann. 1998;27:355–361
24. Task Force on Community Preventive Services. Recommendations re- garding interventions to improve vaccination coverage in children, adolescents, and adults. Am J Prev Med. 2000;18:92–96
25. Briss PA, Rodewald LE, Hinman AR, et al. Reviews of evidence regard- ing interventions to improve vaccination coverage in children, adoles- cents, and adults. Am J Prev Med. 2000;18:97–123
26. Linkins R, Feikema S. Immunization registries: the cornerstone of child- hood immunization in the 21st century. Pediatr Ann. 1998;27:349–354
27. Feikema S, Klevens R, Washington M, Barker L. Extra immunization among US children. JAMA. 2000;283:1311–1340
SLANTED NEW STORIES ABOUT RX

Forty percent of the stories [about new medicines] did not provide quantitative information to back up assertions of a drug’s benefit. Of those that did, 83% reported only the relative benefit and not the absolute benefit. Reporting the former figure tends to make a study’s results sound more dramatic. Most stories about a study on alendronate stated that the drug reduced the frequency of hip fractures caused by osteoporosis by 50% but did not explain that such fractures occurred in 1% of patients who took the drug versus 2% of patients who didn’t . . . The analysis found that when news reports cited “experts,” such individuals often had financial links to the maker of the drug under discussion that weren’t disclosed in the story.
Okie S. Selective reporting on medicines. Washington Post Weekly. June 12, 2000
Submitted by Student
96 IMPACT OF RECORD SCATTERING ON THE MEASUREMENT OF IMMUNIZATION COVERAGE