The 2004 Annual Report to the Office of Management and Budget: Implementing the VFC-AFIX Project: A National Strategy to Improve the Quality of Immunization Practices among VFC Providers May 2005 Background/Introduction: In 1995, Congress directed the Centers for Disease Control and Prevention (CDC) to set guidelines for assessing coverage levels in all public clinics as part of the federal funding for immunization programs. A continuous quality improvement strategy was developed to assess coverage levels and provide insight on how to improve coverage levels. This strategy is known by the acronym “AFIX.” The four components of “AFIX” are Assessment of immunization coverage levels, Feedback of the assessment findings to providers, Incentives to motivate and acknowledge change, and eXchange of information on best practices. This strategy has been documented in the CDC Guide to Community Preventive Services as an effective method to improve and sustain immunization coverage levels. Private provider participation in the VFC program has created a shift in the provision of vaccine services over the last decade from public health clinics to private provider offices. Recognizing this shift, an objective related to the assessment of immunization coverage levels was included in the Healthy People 2010 goals. The objective is to “increase the proportion of providers who have measured the vaccination coverage levels among children in their practice population within the last two years.” The Healthy People 2010 report included a 1997 baseline measurement for providers that had assessed their coverage levels in the last two years as 6% for private providers and 66% for public providers. In 2000, CDC/National Immunization Program (NIP) launched the VFC-AFIX initiative which linked AFIX, the continuous quality improvement strategy, with the VFC program. The year 2004 represents the fourth year that the 61 eligible VFC grantees requested and received funding for the VFC-AFIX project. The purpose of this project is to assess and improve immunization delivery practices at the provider level to assure that VFC-eligible children are receiving quality services. 2004 NIP Programmatic Activities The activities in 2004 enhanced the VFC-AFIX project at the federal and grantee level. The completed and on-going priority activities for NIP/AFIX staff are described below. * The Core Elements of AFIX Training and Implementation was revised and published as a “2nd Edition.” The Core Elements of AFIX Training and Implementation was first released in 2002 for use by grantees in training their staff and implementing the AFIX process at the provider level. This guide was revised and released in 2004 at the National Immunization Conference. The 2nd edition of this document included more information about how to implement AFIX for adult populations and how to use immunization registries for the assessment component of AFIX. Hard copies of the Core Elements were distributed to the grantees in 2004. The Core Elements document 2nd edition is available electronically on the AFIX website http://www.cdc.gov/vaccines/programs/afix/ . A hard copy of the Core Elements document is attached for review. * In 2004, NIP/AFIX staff and grantee Clinic Provider Assessment Workgroup (CPAWG) members developed AFIX Standards for implementation by all grantees. The purpose of AFIX Standards is to provide grantee AFIX staff information and guidance on the process for developing fundamentally sound AFIX programs that meet CDC grant requirements. The Standards include specific components that address the development, implementation and evaluation of an AFIX program at the grantee level. The AFIX Standards also provide NIP staff with the ability to evaluate the grantees’ performances in effectively implementing AFIX programs. The AFIX Standards were officially released at the 2005 National Immunization Conference, and hard copies of the AFIX Standards were mailed to grantees in April 2005. The AFIX Standards also are posted on the AFIX website under “Major Links,” “What’s New?” and “Publications.” The website link to the AFIX Standards document is http://www.cdc.gov/vaccines/programs/afix/stds-guide.htm. A hard copy of the AFIX Standards is attached for review. * VFC-AFIX project staff coordinated, moderated or presented the following workshop sessions at the 2004 National Immunization Conference: * Core Elements of AFIX Two: What’s New? * Preliminary Results of the VFC/AFIX Activities Survey of Program Managers * The Immunization Grant Process: What Does it Really Mean? * Making the Transition: From Chart-Based to Registry-Based Assessment * VFC-AFIX project staff coordinated and facilitated discussions during quarterly conference calls between grantees and NIP staff regarding VFC-AFIX activities. * NIP/VFC-AFIX staff developed an AFIX grant review tool to create a standard process for reviewing the AFIX information provided by the grantees in the 2005 grant applications submitted in the fall of 2004. * Progress slowed on the Comprehensive Clinic Assessment Software Application (CoCASA) project. This interruption was due in part to contract issues that delayed hiring the majority of software programming staff that work on this project. Once completed, CoCASA will combine and improve the functionality of the existing Clinic Assessment Software Application (CASA), the Adult Clinic Assessment Software Application (ACASA), and the VFC-AFIX Evaluation Software. Work conducted on this project in 2004 included advancement of the CoCASA functionality requirements with specific focus on the VFC-AFIX Evaluation software component since this component would be released for use by all grantees in January 2005. When the software is completely finished, it will also include a tutorial that provides instruction on the AFIX process, using CoCASA as a tool for implementing AFIX and using CoCASA to document VFC-AFIX activities. NIP staff worked on the content development for the tutorial and recorded narration for the completed sections of this tutorial. The updated timeline for release of the full software product is late 2005. 2004 Grantee Programmatic Activities Grantees actively worked to improve VFC/AFIX operations in 2004. This section of the report illustrates some of the new or refined VFC/AFIX activities conducted by a sample of grantees in 2004. * Chicago The Chicago VFC-AFIX staff recorded VFC provider site visits in the CDC VFC-AFIX Evaluation software to assist with organizing the VFC/AFIX site visits and producing aggregate results. The site visits were entered monthly and reviewed to develop a follow-up plan for each individual site. * Indiana A provider satisfaction survey was conducted to improve interaction with providers by making the forms easier to understand, reviewing and improving CASA/AFIX practices, increasing the educational material distributed to providers, and evaluating the usefulness of the Vaccine E-letter. The Immunization Program routinely promotes reminder/recall systems at all AFIX site visits. Field representatives discuss reminder/recall during site visits with public providers. * Massachusetts Follow-up activities by partner organizations continued with a particular focus on ensuring implementation of reminder/recall systems. In order to streamline the site visit process and reduce the number of forms to be completed, a comprehensive assessment tool was created, combining the multiple forms previously used into one comprehensive assessment questionnaire. Training was provided in May for all immunization program staff conducting assessments. This training included review of proper vaccine storage and handling, documentation of immunizations, compliance with VFC Program requirements, LQA methodology, and feedback/exchange of information with providers. The Massachusetts immunization program will update and distribute Immunization Tool Kits to all newly assessed primary care provider sites. * Missouri During 2004, Missouri’s Immunization Quality Improvement Workgroup continued to develop and expand the VFC Site Visit and AFIX Policies and Procedures manual. A standardized action plan was developed and incorporated into VFC site and AFIX visits to ensure that visits are conducted in a uniform manner. * Oregon Oregon staff held community provider meetings across the state in cooperation with LHDs. Several of these meetings were titled “AFIX Exchange” and focused on sharing information about AFIX. * San Antonio San Antonio employed AFIX methodologies utilizing CASA to evaluate WIC staff participation and assessment accuracy. These methods helped to identify deficiencies in the WIC staff’s immunization assessment processes and provided valuable feedback to the WIC Director for future improvement efforts. * Tennessee VFC/AFIX trainings were conducted in April and August to an audience of 52 participants. Instruction was given on VFC/AFIX program overview and standards, vaccine ordering, 24 month-old survey, record de-duplication, Sensaphone use, and Tennessee Web Immunization System (TWIS). * Texas The Immunization Branch implemented a review process; every reviewer hired by the contractor will be observed conducting a minimum of one site per year. A review tool has been developed, and feedback is provided to the contractor’s administrative office for coaching/training purposes. * Vermont Development of the AFIX program included creation of a program specific AFIX manual incorporating the CDC AFIX Core Elements. VFC AFIX FY 2004 Awards to Grantees In 2004, a total of $ 16.4 million in VFC AFIX funds was awarded to grantees. See attached table. Summary of Program Findings The AFIX Standards were developed to provide a framework for grantees to standardize their VFC/AFIX programs within their geographic location to improve the quality of the VFC/AFIX provider interaction. A wide variety of challenges continue to face the grantees implementing the VFC/AFIX initiative including grantee imposed travel restrictions, limited staffing and staff turnover. All these factors impact the quality and effectiveness of grantee provider site visits. Analysis of 2004 Grantee Submitted Data CDC/NIP created the VFC/AFIX Evaluation Software as a tool for grantees to monitor their VFC/AFIX activities. This software is a Microsoft ACCESS database that can be used to store the VFC/AFIX site visit data in accordance with the report requirements due at the end of each calendar year. If this software is used by the grantees, then the aggregate data required for submittal to CDC/NIP can be automatically generated by the report options built into the software. The VFC/AFIX Evaluation Software was initially distributed to the grantees in 2002, and a revised copy has been provided annually through 2004. Grantees are not required to use the VFC/AFIX Evaluation Software; however, they are advised to develop their own tracking system that would capture the same data fields if they choose not to use the software. Grantees were required to submit documentation of their 2004 VFC/AFIX activities as part of the Annual VFC Management Survey due March 1, 2005. The Annual VFC Management Survey data were collected using a web-based reporting method. Of the 61 eligible grantees, 59 submitted data for the Annual VFC Management Survey. VFC/AFIX Staff The number of full-time employees (FTEs) working on VFC/AFIX activities is tabulated in Table 1 below. In total, 306 FTEs are employed at the state level, and 158 FTEs are employed at the local level. Together, 464 FTEs at the state or local level are currently working on VFC/AFIX related activities across the country. Additionally, there are 77 contract staff members working on VFC/AFIX related activities at the state or local level. As shown in Table 1, these numbers can be categorized by new and existing positions for calendar year 2004. The grantee VFC/AFIX projects can be carried out at the state level or the local level by immunization program staff or by staff hired through contracts with outside agencies. Table 1. Number of FTEs Working on VFC/AFIX Project CY2004 Personnel Program Staff* Contract Staff** Total State Local State Local State Local New Positions 15 0 7 4 22 4 Existing Positions 291 158 52 14 343 172 Total 306 158 59 18 365 176 *State Program Staff: state-employed staff working on VFC/AFIX at the state immunization program level. Local Program Staff: local health department staff funded with federal funds who work on VFC/AFIX. **State Contract Staff: VFC/AFIX staff hired by the state immunization programs through third party contracts with federal funds. Local Contract Staff: VFC/AFIX staff at local health departments hired through contracts with outside agencies using federal funds. Provider Information Table 2 below includes the number of provider sites that received at least one visit during 2004 for each category of provider. Public providers are divided into three categories. The “Public” category includes local health departments and Indian Health Service clinics. Community or Migrant Health Centers (C/MHC) includes Federally Qualified Health Care Centers, and the “Other Public” category captures all other facilities that are not included in the other two categories. “Private” represents all private providers that received at least one visit in 2004. The providers are categorized into “VFC-Enrolled sites” and “Non-VFC Enrolled sites.” Including both types of provider sites, a total of 24,662 provider sites received at least one visit in calendar year 2004. Of the 24,662 provider sites, 99 % were enrolled in the VFC program. The 24,400 VFC-Enrolled provider sites that were visited constitute 55% of the total active VFC-Enrolled provider sites, an increase from 52% (22,732 provider sites) in 2003. Table 3 illustrates the percentage of visits in the public sector and the private sector. The percentage of visits in the private sector increased from 47% in 2003 to 53% in 2004. Table 2. Number of Provider Sites Receiving at Least One visit, CY2004 Provider Information Public C/MHC* Other Public Subtotal All Public Private Total VFC-Enrolled Provider Sites 3,435 2,672 1,244 7,351 17,049 24,400 Non-VFC Provider Sites 7 13 1 21 241 262 Total 3,442 2,685 1,245 7,372 17,290 24,662 *Community or Migrant Health Center Table 3. Number and Percent of VFC-enrolled Provider Sites Receiving at Least One Visit, CY2004 Provider Information All Public Private Total Number of VFC-Enrolled Provider Sites 11,809 32,292 44,101 Number (Percent) of VFC-Enrolled Providers Who Received at Least One Visit, CY2004 7,351 (62%) 17,049 (53%) 24,400 (55%) Site Visit Information While 24,400 VFC-enrolled provider sites were visited in 2004, the actual number of VFC Only, AFIX Only, VFC/AFIX combined and educational visits to a VFC-enrolled provider site totaled 27,525. Table 4 below details the number of visits to VFC and Non-VFC provider sites by visit type and provider type. As the table shows, 8,864 visits were conducted in public VFC-enrolled sites and 18,661 visits were conducted in private VFC-enrolled sites. In addition, 211 visits (AFIX Only and Educational) were conducted in Non-VFC enrolled provider sites. Table 4. Total Number of Visits by Provider Type, CY2004 Type of Visit Public VFC Enrolled Public Non-VFC C/MHC VFC Enrolled C/MHC Non-VFC Other Public VFC Enrolled Other Public Non-VFC Private VFC Enrolled Private Non-VFC VFC Only* 1,148 606 511 4,455 AFIX Only** 794 1 257 0 127 2 1,356 172 VFC/AFIX Combined*** 1,906 1,731 538 9,370 Educational 573 0 444 0 229 2 3,480 34 * VFC Only is defined as a visit to a VFC enrolled provider to ensure compliance with VFC program requirements. ** AFIX Only is defined as a quality improvement strategy utilizing assessment of immunization records, feedback, incentive, and exchange of information through performance measurement, diagnosis of service delivery problems, and data feedback during a visit to a medical practice. One AFIX visit should contain an assessment and a feedback component even though more than one physical visit to the provider site may be required to complete the assessment and the feedback sessions. *** A VFC/AFIX Combined site visit is defined as a visit to a VFC-enrolled provider site which integrates the review to ensure compliance with VFC program requirements and immunization record assessment and feedback activities. Table 5 includes additional documentation regarding site visits in CY 2004. Repeat AFIX visits are a subset of the AFIX visits documented in Table 4, and they are included to illustrate the number of visits that occurred at a site that had previously received an AFIX visit between January 1, 2000, and December 31, 2003. The nature of AFIX as a continuous quality improvement strategy requires that provider sites are visited on more than one occasion to evaluate incremental progress. The number of repeat AFIX visits allows CDC/NIP to track the grantees’ progress in implementing this ongoing quality improvement strategy. The VFC Follow-Up visits describe the number of visits that occurred as a result of a problem and/or concern found at the initial VFC visit. The information in Tables 4 and 5 reveals that grantees are actively visiting provider sites, following up on problems identified in previous visits, providing education as well as service, and ultimately building relationships with staff. Table 5. Additional Visits, CY2004 Type of Visit Public VFC Enrolled Public Non-VFC C/MHC VFC Enrolled C/MHC Non-VFC Other Public VFC Enrolled Other Public Non-VFC Private VFC Enrolled Private Non-VFC Repeat AFIX* 1,386 0 697 0 119 0 3,098 50 Follow-up VFC** 1,510 613 70 3,006 *Repeat AFIX: the number of AFIX visits (includes AFIX only and Combined VFC/AFIX) from Table 4 that are repeat assessments (e.g. the provider received an assessment during a previous year). **Follow-up VFC: the number of visits completed to evaluate provider response to previously identified problems found during the initial VFC site visit. As part of the annual grant application process, grantees are required to specify the proposed number of site visits to be conducted in the upcoming calendar year. For the 2004 grant applications, the planned number of site visits included three categories: VFC Only, AFIX Only and VFC/AFIX Combined. In an effort to examine the accuracy with which a grantee can estimate VFC/AFIX activities, the number of proposed site visits from the grant applications were compared with the actual number of site visits (for the categories of VFC Only, AFIX Only and VFC/AFIX Combined). Grantees were not able to conduct the total number of proposed site visits in CY 2004 (see Table 6); however, the grantees did exceed the number of visits completed in CY2003 by 762. Table 6. Proposed and Actual Number of Site Visits, CY2004 Type of Site Visit Public Provider* Private Provider Total Visits Proposed Actual Proposed Actual Proposed Actual VFC only 2,682 2,265 4,972 4,455 7,654 6,720 AFIX only 1,265 1,181 1,579 1,528 2,844 2,709 VFC/AFIX combined 5,700 4,175 9,496 9,370 15,196 13,545 Total visits 9,647 7,621 16,047 15,353 25,694 22,974 *Public provider includes Public, C/MHC, and Other Public. Assessment Outcome Measures for Public Providers Of the 61 grantees that receive funds for VFC-AFIX activities, 55 submitted public provider data for the annual report (see Appendix A, Table 1). Of the 55 that submitted data, 47 use the CASA, Mini-CASA or equivalent method (designated as “CASA”). Three grantees use the Hybrid method, and five grantees use a combination of CASA and Hybrid (designated as “Both”). Information on difference between the CASA and Hybrid methods is found in Appendix B of the results section. Grantees that use CASA as the assessment method will receive an estimated immunization coverage level for each provider assessed. Grantees that use the Hybrid assessment method will receive a result indicating the provider is “above” or “below” a pre-determined threshold immunization coverage level. (See Appendix B for more in-depth explanation.) The majority of the grantees (28/55) assess children 24-35 months of age. Other age groups reported include 19-35 months (10 grantees) and 12-23 months (2 grantees); five grantees reported that they assess both ages 12-23 and 24-35 months. Ten grantees responded to this question with “other.” For those grantees using CASA for the assessment, nine grantees reported provider vaccination coverage levels averaging 80% or higher. Seventeen grantees reported provider vaccination coverage levels averaging 70-79 %. Of the remaining results, 16 grantees indicated vaccination coverage levels averaging 60-69%, and ten indicated vaccination coverage levels below 60%. In comparison to the same results in 2003, these outcomes show improvement. In 2003, 14 grantees indicated vaccination coverage levels below 60%. For those grantees using the Hybrid Method, threshold levels of 65, 70, 75, 80 and 90 percent were used. The most common threshold levels were 75, 80 and 90 percent. For those grantees that used the Hybrid methodology, two grantees reported the number of providers above the selected threshold level as greater than 80% of the providers assessed. Note: The above numbers are not mutually exclusive – meaning one grantee could have done some AFIX visits using the Hybrid method and some visits using the CASA, Mini-CASA, or Equivalent method. Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5 Assessment Outcome Measures for Private Providers Of the 61 grantees that receive funds for VFC-AFIX activities, 55 submitted private provider data for the annual report (see Appendix A, Table 2). Of the 55 that submitted data, 47 use the CASA, Mini-CASA or Equivalent method (designated as “CASA”). Three grantees use the Hybrid method, and five grantees use a combination of CASA and Hybrid (designated as “Both”). A large number of the grantees (28/55) assess children 24-35 months of age. Other age groups reported include 19-35 months (11 grantees) and 12-23 months (2 grantees); and five grantees reported that they assess children ages 12-23 and 24-35 months of age. Nine grantees responded to this question with “other.” For those grantees using CASA for the assessment, ten grantees reported provider vaccination coverage levels averaging 80% or higher. Sixteen grantees reported provider vaccination coverage levels averaging 70-79%. For the remainder, 15 grantees indicated vaccination coverage levels averaging 60-69%, and 11 indicated vaccination coverage levels below 60%. For those providers using the Hybrid method, threshold levels of 65, 70, 75, 80 and 90 percent were used. The most common threshold levels were 75 and 80 percent. Three grantees using the Hybrid Method reported 80% of the providers assessed as above the designated threshold level; one of whom reported 100% of the providers assessed as above the 90% designated threshold level. Note: The above numbers are not mutually exclusive – meaning one grantee could have done some AFIX visits using the Hybrid method and some visits using the CASA, Mini-CASA, or Equivalent method. Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5 Change in coverage when using CASA to assess public providers Forty-two grantees used the CASA method to assess coverage during previous and CY2004 assessments in the public sector (see Appendix A, Table 3). The majority (32 of 42 ) assessed for 4 diphtheria, tetanus, and pertussis (DTaP) doses; 3 polio doses; 1 measles, mumps and rubella dose (MMR), 3 haemophilus influenzae type b doses (Hib) and 3 hepatitis B doses (HepB) for previous and CY 2004 assessments. This series of vaccinations is documented as 4:3:1:3:3. In addition, six grantees reported change in coverage levels for an immunization series that included the varicella vaccine. (The 4:3:1:3:3:1 series includes 4 DTaP doses; 3 polio doses; 1 MMR, 3 Hib, 3 HepB, and 1 dose of varicella.) Three grantees assessed for 4 doses of DTaP, 3 doses of polio and 1 dose of MMR for previous and CY 2004 assessments; commonly referred to as the 4:3:1 series. One grantee assessed for the 3:2:2:2 series for previous and CY 2004 assessments, which is 3 doses of DTaP, 2 doses of polio, 2 doses of Hib and 2 doses of HepB at 12 months of age. Twelve grantees documented an increase in average public provider vaccination coverage levels to be 10% or greater compared to eight grantees in 2003. Twenty grantees found the average public provider vaccination levels improved 1-9% as compared to 12 grantees in 2003. The average public provider vaccination coverage levels did not change for one grantee. Nine grantees reported the change in average public provider vaccination as a negative value in 2004 as compared to 17 grantees in 2003. The negative values for the change in coverage only ranged from -1% to -8% for seven of the nine grantees, and -13% and -16% for each of the other two. This is an improvement when compared to the 2003 results which included a negative value change in coverage ranging from -1% to -27.3%. One grantee also documented a change in coverage levels among public providers using the Hybrid method. This grantee reported an increasing percentage of providers who met the designated 80% threshold level from 43% to 51%. Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5. Values between -0.4 and 0.4 were defined as 0 values and interpreted as sites with no change. Change in coverage when using CASA to assess private providers Thirty-nine grantees used the CASA method to assess private provider coverage levels during previous and CY2004 assessments in the private sector (see Appendix A, Table 3). The majority (31 of 39) assessed for the 4:3:1:3:3 series for previous and CY 2004 assessments. Five assessed for the 4:3:1:3:3:1 series and three grantees assessed for the 4:3:1 series. Eleven grantees documented change in the average private provider vaccination level to be 10% or greater as compared to five grantees in 2003. Nineteen grantees found the average public provider vaccination level to change between 1-9%. Overall, more than half of the grantees that submitted 2004 data (n=30) reported an increase in the average private provider vaccination levels compared to only 21 grantees reporting an increase in 2003. The average private provider vaccination coverage levels did not change for three grantees. The change was negative for six grantees in comparison to the 16 grantees that reported negative change in 2003. This negative change ranged from -1 % to -7%. Two grantees documented a change in coverage levels among private providers using the Hybrid method. Both grantees reported an increasing number of providers who met the designated 80% threshold level. One grantee reported an increase in the percentage of providers that met the threshold from 64% to 67% and the other grantee reported an increase from 56% to 87% of providers. Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5. Values between -0.4 and 0.4 were defined as 0 values and interpreted as sites with no change. 2004 NIP Training and Education Activities In 2004, NIP staff working on the VFC/AFIX project limited the number of trainings in 2004 to focus on the development of a training curriculum for a series of National AFIX trainings in the summer of 2005 that would incorporate training on the new CoCASA software. Due to delays beyond the control of AFIX/NIP staff, the completion date of the CoCASA was extended to late 2005. This delay necessitated that the National AFIX trainings be postponed. During 2004, NIP staff conducted on-site AFIX and/or CASA trainings to the following grantees: * Mississippi * Kansas * Missouri * Regional Training for Oregon, Washington and Alaska In addition conference calls were held with the following grantees to provide guidance on revising, restructuring and/or improving their respective AFIX programs: * Georgia * Michigan * Tennessee APPENDIX A: Tables Table A-1: Assessment Outcome Measures for Public Providers by Grantee, CY2004 Grantee Age Group Assessed (Months) (1) Assessment Method (2) Vaccination Series Measured with CASA (3) Number Assessed with CASA (4) Minimum Coverage Level (5) Maximum Coverage Level (6) Crude Average Coverage Level (7) Vaccination Series Assessed with Hybrid (8) Number Assessed with Hybrid (9) Hybrid Threshold Level (10) Number of Providers At or Above Threshold Level (11) Alabama 19-35 casa 4:3:1:3:3 32   100 68.2         Alaska Other casa 4:3:1:3:3 27 46 100 76         Arizona Other casa 4:3:1:3:3 24 0 97 68.5         Arkansas 24-35 casa 4:3:1:3:3 93 44 100 77         California Other both 4:3:1:3:3 331 0 100 67.6 4:3:1:3:3 2 80 1 Chicago 24-35 both 4:3:1:3:3 40 21 96 52.6 4:3:1:3:3 4 75 4 Colorado 19-35 both 4:3:1:3:3 22 22.2 99.9 73.2 4:3:1:3:3 36 65 21 Connecticut 24-35 casa 4:3:1:3:3:1 28 36.5 100 81.3         District of Columbia 19-35 casa 4:3:1:3:3 20 38 88 69         Florida 24-35 casa 4:3:1:3:3 121 33 100 78.7         Georgia 24-35 casa 4:3:1:3:3:1 191 0 100 76         Guam 24-35 casa 4:3:1:3:3 1 0 0 76         Hawaii 24-35 casa 4:3:1:3:3 9 27 89 61.2         Idaho 19-35 casa 4:3:1:3:3 27 40 100 83.3         Illinois 12 & 24 casa 4:3:1:3:3:1 57 0 88.9 44         Indiana 19-35 casa 4:3:1:3:3 169 0 100 82.5         Iowa Other casa 4:3:1:3:3 144 27 100 88.7         Kansas 12 & 24 casa 4:3:1:3:3 127 15 100 79.3         Kentucky Other casa 4:3:1:3:3:1 143 0 97 75.8         Louisiana Other both 4:3:1 116 19.2 100 63.3 4:3:1 30 90 11 Maine 24-35 casa 4:3:1:3:3:1 63   100 45.6         Maryland 24-35 casa 4:3:1:3:3:1 29 54.5 100 78.8         Massachusetts 24-35 hybrid         4:3:1:3:3 55 80 28 Michigan 19-35 hybrid         4:3:1:3:3:1 218 70 105 Minnesota 24-35 casa 4:3:1:3:3 18 17 100 66         Mississippi Other casa 4:3:1 282 0 100 78.3         Missouri 12 & 24 casa 4:3:1:3:3:1 176 6.7 95 53.3         Grantee Age Group Assessed (Months) (1) Assessment Method (2) Vaccination Series Measured with CASA (3) Number Assessed with CASA (4) Minimum Coverage Level (5) Maximum Coverage Level (6) Crude Average Coverage Level (7) Vaccination Series Assessed with Hybrid (8) Number Assessed with Hybrid (9) Hybrid Threshold Level (10) Number of Providers At or Above Threshold Level (11) Montana 24-35 hybrid         4:3:1:3:3 65 90 65 Nebraska 19-35 both 4:3:1:3:3 54 36.4 100 77.3 4:3:1:3:3 5 75 0 Nevada 24-35 casa 4:3:1:3:3 21 35.2 80 58.2         New Hampshire 24-35 casa 4:3:1:3:3 43 0 100 93         New Jersey 24-35 casa 4:3:1 23 40.4 100 73.2         New Mexico 12 & 24 casa 4:3:1:3:3:1 29 33 100 75         New York State 24-35 casa 4:3:1:3:3 77 0 100 67.2         New York City 24-35 casa 4:3:1:3:3 17 43 100 69         North Carolina 24-35 casa 4:3:1:3:3 100 12 96 74         North Dakota 24-35 casa 4:3:1:3:3 20 77.8 100 91.9         Ohio 24-35 casa 4:3:1:3:3 77 22.4 94.4 64.5         Oklahoma 24-35 casa 4:3:1:3:3:1 47 0 85 53.1         Oregon 12 & 24 casa 4:3:1:3:3 53 33.3 76.5 58.6         Pennsylvania 24-35 casa 4:3:1:3:3 65 81.2 84.2 82.7         Philadelphia 19-35 casa 4:3:1:3:3 18 0 100 76         Puerto_Rico Other casa 4:3:1:3:3:1 216 0 100 46         Rhode Island 12-23 casa 3:2:2:2 2 86 89 87.5         San Antonio Other casa 4:3:1:3:3 18 0 100 83.5         South Carolina 24-35 casa 4:3:1:3:3 30 19.3 98 62.5         Tennessee 24-35 casa 4:3:1:3:3:1 65 125 100 57.5         Texas Other casa 4:3:1 673 0 100 67         Utah 24-35 casa 4:3:1:3:3 51 9.4 96.6 60.7         Vermont 19-35 casa 4:3:1:3:3 20 25 100 68.5         Virginia 12-23 casa 4:3:1:3:3:1 25 13 90 53.8         Washington 19-35 casa 4:3:1:3:3 30 20 100 61.2         West Virginia 24-35 casa 4:3:1:3:3 31 29 100 69.1         Wisconsin 24-35 casa 4:3:1:3:3 5 56.3 93.7 76.2         Wyoming 24-35 casa 4:3:1:3:3:1 546     75         Although Delaware, South Dakota, the Mariana Islands and the Virgin Islands conducted site visits, they did not report any measured assessment outcomes prior to the deadline for this report. Houston and American Samoa did not report any data for this survey. 1. Age Group Assessed: the age range of the patients included in the assessment 2. Assessment Method: method used to evaluate vaccine coverage level (CASA or Hybrid) 3. Vaccination Series Measured: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 4. Number Assessed with CASA: number of providers that received an assessment using the CASA method. 5. Minimum Coverage Level: among all providers assessed, the lowest determined vaccination coverage for the vaccination series measured 6. Maximum Coverage Level: among all providers assessed, the highest determined vaccination coverage for the vaccination series measured 7. Crude Average Coverage Level: an unweighted average vaccination coverage level among all provider sites 8. Vaccination Series Assessed with Hybrid: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit when using the Hybrid Method. 9. Number Assessed with Hybrid: number of providers that received an assessment using the CASA method. 10. Hybrid Threshold Level: vaccination coverage level at which providers are expected to be performing at or above 11. Number of Providers At or Above Threshold Level: number of providers at or above threshold level Table A-2: Assessment Outcome Measures for Private Providers by Grantee, CY2004 Grantee Age Group Assessed (Months) (1) Assessment Method (2) Vaccination Series Measured with CASA (3) Number Assessed with CASA (4) Minimum Coverage Level (5) Maximum Coverage Level (6) Crude Average Coverage Level (7) Vaccination Series Assessed with Hybrid (8) Number Assessed with Hybrid (9) Hybrid Threshold Level (10) Number of Providers At or Above Threshold Level (11) Alabama 19-35 casa 4:3:1:3:3 148   100 70.4         Alaska Other casa 4:3:1:3:3 11 0 82 64         Arizona Other casa 4:3:1:3:3 90 0 94 60.6         Arkansas 24-35 casa 4:3:1:3:3 127 0 100 50         California Other both 4:3:1:3:3 35 20 100 70.1 4:3:1:3:3 198 80 138 Chicago 24-35 hybrid 4:3:1:3:3 95 11 100 58.9 4:3:1:3:3 57 75 32 Colorado 19-35 both         4:3:1:3:3 37 65 30 Connecticut 24-35 casa 4:3:1:3:3:1 101 0 100 80.8         Delaware 19-35 casa 4:3:1:3:3 99 8 100 66         District of Columbia 19-35 casa 4:3:1:3:3 39 60 100 67         Florida 24-35 casa 4:3:1:3:3 466 0 100 76.7         Georgia 24-35 casa 4:3:1:3:3:1 194 0 100 83.9         Guam 24-35 casa 4:3:1:3:3 7 30 64 47         Hawaii 24-35 casa 4:3:1:3:3 54 6 100 65.9         Idaho 19-35 casa 4:3:1:3:3 93 0 100 72.2         Illinois 12 & 24 casa 4:3:1:3:3:1 24 0 79 38         Indiana 19-35 casa 4:3:1:3:3 519 0 100 82.6         Iowa Other casa 4:3:1:3:3 85 50 100 81.8         Kansas 12 & 24 casa 4:3:1:3:3 52 0 93 54.4         Kentucky Other casa 4:3:1:3:3:1 37 10 94 73.3         Louisiana Other both 4:3:1 153 0 100 70.8 4:3:1 45 90 16 Maine 24-35 casa 4:3:1:3:3:1 162   100 47.4         Maryland 24-35 casa 4:3:1:3:3:1 550 15.4 100 87.8         Grantee Age Group Assessed (Months) (1) Assessment Method (2) Vaccination Series Measured with CASA (3) Number Assessed with CASA (4) Minimum Coverage Level (5) Maximum Coverage Level (6) Crude Average Coverage Level (7) Vaccination Series Assessed with Hybrid (8) Number Assessed with Hybrid (9) Hybrid Threshold Level (10) Number of Providers At or Above Threshold Level (11) Massachusetts 24-35 hybrid         4:3:1:3:3 254 80 169 Michigan 19-35 both 4:3:1:3:3:1 187 22 99 64 4:3:1:3:3:1 535 70 317 Minnesota 24-35 casa 4:3:1:3:3 93 14 98 65         Mississippi Other casa 3:2:2:2 140 23 100 76.1         Missouri 12 & 24 casa 4:3:1:3:3:1 180 0 100 59.7         Montana 24-35 hybrid         4:3:1:3:3 58 90 58 Nebraska 19-35 both 4:3:1:3:3 61 0 100 82.2 4:3:1:3:3 54 75 45 Nevada 24-35 casa 4:3:1:3:3 29 0 100 56.2         New Hampshire 24-35 casa 4:3:1:3:3 83 50 100 85         New Jersey 24-35 casa 4:3:1 3 52 62 57.3         New Mexico 12 & 24 casa 4:3:1:3:3:1 117 0 100 70         New York State 24-35 casa 4:3:1:3:3 184 0 100 75.7         New York City 24-35 casa 4:3:1:3:3 135 0 100 59         North Carolina 24-35 casa 4:3:1:3:3 256 0 100 69         North Dakota 24-35 casa 4:3:1:3:3 26 45 100 89.1         Ohio 24-35 casa 4:3:1:3:3 313     63.62         Oklahoma 24-35 casa 4:3:1:3:3:1 55 0 100 39.1         Oregon 12 & 24 casa 4:3:1:3:3 147 0 100 57.26         Pennsylvania 24-35 casa 4:3:1:3:3 346 83.5 85.3 84.4         Philadelphia 19-35 casa 4:3:1:3:3 105 0 100 72         Rhode Island 12-23 casa 3:2:2:2 25 0 100 84.4         San Antonio Other casa 4:3:1:3:3 98 0 100 73.3         South Carolina 24-35 casa 4:3:1:3:3 32 16.7 100 70.8         Tennessee 24-35 casa 4:3:1:3:3:1 126 0 100 68.3         Texas Other casa 4:3:1 1795 0 100 70         Utah 24-35 casa 4:3:1:3:3 160 0 100 58.6         Vermont 19-35 casa 4:3:1:3:3 83 0 100 68.98         Virginia 12-23 casa 4:3:1:3:3:1 118 0 100 65.64         Washington 19-35 casa 4:3:1:3:3 95 17 100 65.64         West Virginia 24-35 casa 4:3:1:3:3 38 14 100 76.4         Wisconsin 24-35 casa 4:3:1:3:3 55 0 100 69.7         Wyoming 24-35 casa 4:3:1:3:3:1 400     77         Although South Dakota, the Mariana Islands, Puerto Rico, and the Virgin Islands conducted site visits, they did report any measured assessment outcomes prior to the deadline for this report. Houston and American Samoa did not report any data for this survey. 1. Age Group Assessed: the age range of the patients included in the assessment 2. Assessment Method: method used to evaluate vaccine coverage level (CASA or Hybrid) 3. Vaccination Series Measured with CASA: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 4. Number Assessed with CASA: number of providers that received an assessment using the CASA method. 5. Minimum Coverage Level: among all providers assessed, the lowest determined vaccination coverage for the vaccination series measured 6. Maximum Coverage Level: among all providers assessed, the highest determined vaccination coverage for the vaccination series measured 7. Crude Average Coverage Level: an unweighted average vaccination coverage level among all provider sites 8. Vaccination Series Assessed with Hybrid: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit when using the Hybrid Method. 9. Number Assessed with Hybrid: number of providers that received an assessment using the CASA method. 10. Hybrid Threshold Level: vaccination coverage level at which providers are expected to be performing at or above 11. Number of Providers At or Above Threshold Level: number of providers at or above threshold level Table A-3: Change in Vaccination Coverage Levels, Grantees That Used CASA Method for Previous and CY2004 Assessments Grantee Total Number of Providers Assessed (1) Vaccination Series Measured (2) Crude Average Coverage Level (Previous) (3) Crude Average Coverage Level (CY2004) (4) Average Percentage Point Change in Coverage (5) Crude Average Coverage Level (Previous) (3) Crude Average Coverage Level (CY2004) (4) Average Percentage Point Change in Coverage (5) Alaska 38 4:3:1:3:3 77 76 -1 60 64 4 Arizona 22 4:3:1:3:3 72 75.4 3.4 58.9 61.6 2.7 Arkansas 94 4:3:1:3:3 94 93 -1       California* 284 4:3:1:3:3 64.5 68.5 4       Chicago 269 4:3:1:3:3     40 66.7 26.7 Connecticut 52 4:3:1:3:3:1 71 79.8 8.8 75.2 78.2 3 District of Columbia 54 4:3:1:3:3 54 69 15 58 67 9 Florida   4:3:1:3:3 75 80.4 5.4 73.2 77.7 4.5 Georgia 202 4:3:1:3:3:1 82.7 74.4 -8.2 86.6 87 0.4 Hawaii 10 4:3:1:3:3 58.6 61 2.4 69 70.7 1.7 Idaho 124 4:3:1:3:3 82.5 80.3 -2.2 70 72 2 Indiana 680 4:3:1:3:3 77.5 82.5 5 78.4 82.6 4.2 Iowa 14 4:3:1:3:3 60.3 79.7 19.4 65.6 75.5 9.9 Kansas 117 4:3:1:3:3 65 81.1 16.1 59.6 59.9 0.3 Kentucky 246 4:3:1:3:3 71.2 83.4 12.2 71.8 79.4 7.6 Louisiana 175 4:3:1 56.4 62.8 6.4 67.1 77 9.9 Maine 151 4:3:1:3:3 76.2 60 -16.2 70.8 63.4 -7.4 Massachusetts*   4:3:1:3:3             Michigan 84 4:3:1:3:3       60 69 9 Minnesota 53 4:3:1:3:3 62 63 1 70 67 -3 Mississippi 422 4:3:1 77.7 78.6 0.9 76 71.4 -4.6 Missouri 37 3:2:2:2 56.5 77 20.5   Grantee Total Number of Providers Assessed (1) Vaccination Series Measured (2) Crude Average Coverage Level (Previous) (3) Crude Average Coverage Level (CY2004) (4) Average Percentage Point Change in Coverage (5) Crude Average Coverage Level (Previous) (3) Crude Average Coverage Level (CY2004) (4) Average Percentage Point Change in Coverage (5) Nebraska 73 4:3:1:3:3 72 76.5 4 72 80.3 8.3 Nevada 32 4:3:1:3:3 60.2 58.4 -1.9 66 64.4 -1.6 New Hampshire 116 4:3:1:3:3 82 85 3 83 84 1 New Mexico 95 4:3:1:3:3:1 58 75 17 58 73 15 New York State 195 4:3:1:3:3 65.4 68.4 3 76.1 74.4 -1.7 New York City 60 4:3:1:3:3 68 70 2 46 58 12 North Carolina 222 4:3:1:3:3 72 74 2 72 74 2 North Dakota 5 4:3:1:3:3 86.7 83 -3.7 35.7 90.7 55 Ohio 75 4:3:1:3:3 57.4 64.5 7.1       Oklahoma 103 4:3:1:3:3:1 47.3 62.6 15.3 42 60.2 18.2 Oregon 200 4:3:1:3:3 56.8 58.6 1.8 58.8 57.3 -1.5 Pennsylvania   4:3:1:3:3 83 82.7 -0.3 82.1 84.4 2.3 Philadelphia 109 4:3:1:3:3 72 76 4 70 72 2 Puerto_Rico 196 4:3:1:3:3:1 53 49 -4 San Antonio 116 4:3:1:3:3 66.4 86.9 20.5 61.3 73.3 12 South Carolina 25 4:3:1:3:3 29.7 61.4 31.7 55.6 69.2 13.6 Tennessee 39 4:3:1:3:3:1 57.4 66.7 9.3 63.9 72.2 8.3 Texas 1949 4:3:1 57 67 10 59.1 70 11 Utah 156 4:3:1:3:3 78 64.5 -13.4 54.3 62.9 8.6 Virginia 260 4:3:1:3:3 57.2 61.9 4.7 73.1 75.5 2.4 Washington 193 4:3:1:3:3 49.5 61.2 11.7 54.5 65.6 11.1 West Virginia 55 4:3:1:3:3 63.1 70 6.9 66.3 74 7.7 Wisconsin 91 4:3:1:3:3 38.1 93.7 55.6 100 100 0 * Grantee provided change in coverage levels for public and/or private providers using hybrid assessment methodology. See text for assessment results. 1Total Number Assessed: total number of providers that received an AFIX visit 2 Vaccination Series Measured: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 3 Crude Average Coverage Level (Previous): an unweighted average vaccination coverage level among all provider sites based on vaccination coverage levels from assessments performed between 1/01/01 and 12/31/02 4 Average Coverage Level (CY 2002): an unweighted average vaccination coverage level among all provider sites based on vaccination coverage levels from assessments performed during CY 2003 5 Average Percentage Point Change in Coverage: increase or decrease between previous coverage level and coverage level CY 2002 APPENDIX B: Assessment Methods Appendix B: Assessment Methodology Options Method Description Advantages Disadvantages Diagnostic Tools CASA Sample selection: random sample Sample size: approximately 100 per cohort Inputs: child’s demographic information; date of each immunization; other information related to diagnostic analysis Outputs: diagnostic information on late starts, drop-offs and missed opportunities baseline measure of coverage Precise estimates of immunization coverage levels; evaluation of missed opportunities; evaluation of late starts, etc. Large sample size Resource burden (staff, time) Registry based Sample selection: census of all eligible records Sample size: all eligible records Inputs: data from registry downloaded to CASA Outputs: diagnostic information on late starts, drop-outs and missed opportunities Minimal time and effort for data collection no sampling error since estimates based on census of records. Potential biases: - Registry may not contain 100% of provider’s records - Reliability of registry data Mini-CASA Sample selection: consecutive, convenience or random sample Sample size: 40-60 Inputs: same as CASA Outputs: diagnostic information on late starts, drop-off and missed opportunities Smaller sample sizes Coverage estimates have less precision; sample may not be randomly selected Hybrid Hybrid (LQA/CASA) Sample selection: random sample Sample size: 30 Inputs: same as CASA Outputs: determines if a provider has immunization coverage above or below a specified threshold vaccine histories of not up-to-date clients as examples to discuss with provider and staff Smallest sample size Rapid assessment More feedback information than LQA alone Identifies providers who may benefit from a diagnostic assessment Computer needed (not paper and pencil as LQA) Does not give point estimate of coverage Smaller basis for diagnostic feedback 1