The 2005 Annual Report to the Office of Management and Budget: Implementing the VFC-AFIX Project: A National Strategy to Improve the Quality of Immunization Practices among VFC Providers May 2006 Executive Summary The VFC-AFIX Activities in 2005 included two major milestones: the formal release of the AFIX Standards, Level I and the availability of the VFC-AFIX Evaluation Module of Comprehensive Clinic Assessment Software Application (CoCASA). The AFIX Standards are designed to assist grantees in the formalization of AFIX processes that will produce more quality improvement focused activities and outcomes. The VFC-AFIX Evaluation Module of CoCASA is also designed to assist grantees, particularly in expediting the creation of reports that summarize the VFC-AFIX activities and outcomes from the previous year. In 2005, $16,499,324 in VFC/AFIX funds was awarded to grantees. Grantees were required to submit documentation of their 2005 VFC/AFIX activities as part of the Annual VFC Management Survey due March 1, 2006. The Annual VFC Management Survey data were collected using a web-based reporting method. Of the 61 eligible grantees, 61 submitted data for the Annual VFC Management Survey. Ninety-nine percent (22,910 of 23,200) of the provider sites that were visited at least once in 2005 were enrolled in the VFC program. These 22,910 VFC-enrolled provider sites constitute 51% of the total active VFC-enrolled provider sites. The 22,910 VFC-enrolled provider sites visited at least once in 2005 received a total of 29,809 visits. These visits include VFC-enrollment visits, VFC Only, AFIX Only, VFC/AFIX combined, and educational visits. Five hundred thirty-four FTEs at the state (275) and local (259) level are currently working on VFC/AFIX related activities across the country. Additionally, there are 98 contract staff members working on VFC/AFIX related activities at the state or local level. The data are self-reported by the grantee and NIP staff has realized through the data cleaning process that more education and technical assistance must be provided to the grantees throughout 2006 as they document their VFC and AFIX activities (and outcomes). The primary focus will be to consistently and methodically categorize activities and document them using standardized protocols. The AFIX Standards Level I should assist the grantees in the development of these standard protocols; NIP staff’s responsibility will be to ensure they are implemented appropriately. In the following year, it is anticipated that increased familiarity with the CoCASA software as well as increased technical assistance from NIP will improve the quality of the data. The 2005 Annual Report to the Office of Management and Budget: Implementing the VFC-AFIX Project: A National Strategy to Improve the Quality of Immunization Practices among VFC Providers May 2006 Background/Introduction: In 1995, Congress directed the Centers for Disease Control and Prevention (CDC) to set guidelines for assessing coverage levels in all public clinics as part of the federal funding for immunization programs. A continuous quality improvement strategy was developed to assess coverage levels and provide insight on how to improve coverage levels. This strategy is known by the acronym “AFIX.” The four components of “AFIX” are Assessment of immunization coverage levels, Feedback of the assessment findings to providers, Incentives to motivate and acknowledge change, and eXchange of information on best practices. This strategy of practice-based assessment has been recommended by ACIP, the National Vaccine Advisory Committee, the American Academy of Pediatrics, and the American Academy of Family Physicians, as well as the Task Force for Community Preventive Services. Private provider participation in the Vaccines For Children (VFC) program has created a shift in the provision of vaccine services over the last decade from public health clinics to private provider offices. An objective related to the assessment of immunization coverage levels was included in the Healthy People 2010 goals. The objective is to “increase the proportion of providers who have measured the vaccination coverage levels among children in their practice population within the last two years,” and there are specific goals for both public sector and private sector providers. In 2000, CDC/National Immunization Program (NIP) launched the VFC-AFIX initiative which linked AFIX, the continuous quality improvement strategy, with the VFC program. The year 2005 represents the fifth year that the 61 eligible VFC grantees requested and received funding for the VFC-AFIX project. The purpose of this project is to assess and improve immunization delivery practices at the provider level to assure that VFC-eligible children are receiving quality services. 2005 NIP Programmatic Activities The activities in 2005 enhanced the VFC-AFIX project at the federal and grantee level. Many of the activities that NIP focused on in 2005 were related to finalizing Comprehensive Clinic Assessment Software Application (CoCASA), immunization assessment software being developed by NIP to assist grantees with the “Assessment” component of AFIX. In addition to the assessment capabilities, the software contains an evaluation module for VFC and AFIX activities. This evaluation module allows grantees to utilize CoCASA to enter the providers’ responses to the VFC site visit questionnaire and document results of the AFIX visit; the module also aggregate reports that assist the grantees in completing the VFC Management Survey due annually to NIP on March 1st. The VFC Management Survey collects information on each grantee’s VFC and AFIX activities for the previous year. This component of the software was completed and released in January 2005 to allow grantees to record the results of the VFC and AFIX site visits for the entire calendar year of 2005. Below are highlights of key activities conducted by NIP related to VFC and AFIX. * NIP staff working on VFC/AFIX activities developed and conducted a web seminar on the VFC/AFIX Evaluation Module of CoCASA on January 26, 2005. This seminar was archived and made available on the AFIX website for reference through January 2006. * The AFIX Standards were officially released at the 2005 National Immunization Conference, and hard copies of the AFIX Standards were mailed to grantees in April 2005. The AFIX Standards also are posted on the AFIX website under “Major Links,” “What’s New?” and “Publications.” The website link to the AFIX Standards document is http://www.cdc.gov/vaccines/programs/afix/stds-guide.htm * NIP staff coordinated, moderated or presented the following workshop sessions at the 2005 National Immunization Conference: ( Registries as Assessment Tools to Improve AFIX Efficiency ( AFIX Standards: What are They and How Do I Use Them? ? Improving Your Vaccines for Children (VFC)/AFIX Program to Increase Childhood Immunization Rates ? AFIX Ad Hoc Meeting on CoCASA and the VFC/AFIX Evaluation Module ? AFIX lunch rounds table * NIP staff working on VFC/AFIX activities developed the content for the training component of CoCASA. The content included tutorials on the AFIX process and the functions of CoCASA including the VFC/AFIX Evaluation Module of the software. In addition to developing the content, staff worked with graphic designers to create the automated tutorials. * NIP staff coordinated and facilitated discussions during quarterly conference calls between grantees and NIP staff regarding VFC-AFIX activities. * NIP staff developed a simplistic model to determine the cost of AFIX visits based on how AFIX is implemented at the grantee level. The implementation methods were based on who implemented AFIX at the grantee level. The grantees use four different staffing methods to implement AFIX at the grantee level: Grantee staff implemented (N=29 grantees), Contract Staff implemented (N=4 grantees), Combination Method (N=23 grantees) and Local Health Department implemented (N=1 grantee). * NIP staff provided technical assistance onsite or via telephone to the following grantees on AFIX Standards or other AFIX related issues: ( Wisconsin ? Louisiana ? Michigan ? Nevada ? Colorado * NIP staff developed the scripts used for the Function and System Testing (FST) of the full CoCASA. Additionally, staff recruited “testers” from grantees VFC/AFIX programs to participate in the FST process prior to the release of CoCASA. * NIP staff working on the VFC/AFIX activities chaired and actively participated on the workgroup that developed the content, recruited presenters and implemented “Vaccine University” training. This training focused on three key programs or processes for grantees: VFC, AFIX and Vaccine Management. The target audience for the training was grantee staff members who provide daily oversight to these programs at the grantee or local levels. In addition to coordinating the overall training, NIP’s VFC/AFIX staff developed and presented material in all six breakout sessions and one concurrent plenary session. The focus for the AFIX track was to orient the grantees on CoCASA in context of the overall AFIX process. Vaccine University was selected as NIP’s training success story for 2005. Please see Attachments A, B, and C for a copy of Vaccine University’s Student Handbook (Agenda), summary of participant’s training evaluations, and NIP’s 2005 Annual Report spotlighting Vaccine University. * NIP staff provided training on adult CASA to a staff member from the American College of Physicians who is working on an Adult flu AFIX project. 2005 Grantee Programmatic Activities Grantees actively worked to improve VFC/AFIX operations in 2005. This section of the report illustrates some of the new or refined VFC/AFIX activities conducted by a sample of grantees in 2005. * Sixteen Grantees reported actively working towards achieving AFIX Standards Level I. * Wisconsin achieved the AFIX Standards Level I and presented at Vaccine University on how they developed their state policy manual based on the AFIX Standards Level I. * Florida provided certificates of achievement to community and county health organizations with high immunization coverage levels. This incentive effort was a pilot and will be reviewed for possible replication. * Hawaii presented awards to providers that achieved immunization coverage levels of 90% or greater based on the results of their AFIX visits. * Louisiana assembled immunization tool kits to be sent to all newly assessed primary care providers; and updated packets are sent to previously assessed provider sites and vaccine distribution sites. * Minnesota featured an article on the benefits of VFC/AFIX visits in “Got Your Shots” bi-monthly newsletter that is distributed to all VFC enrolled clinics to encourage provider participation in the program. * New York State collaborated with Office of Managed Care and the Office of Medicaid. * Ohio AFIX staff presented at Vaccine University on collaboration activities between AFIX and private provider organizations. * Houston conducted repeat AFIX visits to 25 providers and reported an increase of 16.4% on the coverage level for the 4-3-1-3-3-1 series. * Utah continued their collaboration with Intermountain Health Care on AFIX and began Adult AFIX in Community Health Clinics. * West Virginia developed certificates to be presented to public health providers at their annual conference and has drafted plans for recognition of private providers at the West Virginia Infectious Disease Conference. Best practices for immunization coverage level improvement are shared among providers. Other Grantee Quality Assurance Activities In addition to AFIX, activities many other quality assurance activities are implemented at the grantee level. Many of these activities compliment and enhance AFIX activities at the grantee level. This section of the report illustrates some of these other Quality Assurance activities occurring at the grantee level: * Michigan promotes and provides educational interventions on vaccine-preventable diseases and immunizations to a minimum of 1,500 healthcare professionals in the state. * Washington state continued the Educating Physicians In their Community-based immunization practitioner training project, Training on Immunizations for Practitioners and Staff (TIPS) through a contract with Public Health Seattle/ King County. * The Utah Immunization Program provides staff support to state coalitions: Every Child by Two, Utah Adult Immunization Coalition, and Utah Vaccine Advisory Committee; and two local coalitions: Salt Lake Immunization Coalition and the Northern Utah Immunization Coalition. Each coalition is active and working to support and address a variety of immunization issues across the lifespan. * North Carolina conducts educational visits to new private providers who signed up to receive publicly funded vaccine to provide education and ensure compliance with all VFC program requirements. * Indiana Immunization Education staff members conduct Immunization A-Z training sessions and educational programs at nursing schools to expand the knowledge base of nursing students. VFC AFIX FY 2005 Awards to Grantees In 2005, $16,499,324 in VFC AFIX funds was awarded to grantees (see Appendix A). Summary of Program Findings The AFIX/VFC program at both the federal and grantee levels was very active in developing, implementing and evaluating several new strategies to improve the VFC/AFIX process. A wide variety of challenges continue to face the grantees implementing the VFC/AFIX initiative including grantee imposed travel restrictions, limited staffing and staff turnover. All these factors impact the quality and effectiveness of grantee provider site visits. Analysis of 2005 Grantee Submitted Data Grantees were required to submit documentation of their 2005 VFC/AFIX activities as part of the Annual VFC Management Survey due March 1, 2006. The Annual VFC Management Survey data were collected using a web-based reporting method. Of the 61 eligible grantees, 61 submitted data for the Annual VFC Management Survey. CDC/NIP created the original VFC/AFIX Evaluation Software as a tool for grantees to monitor their VFC/AFIX activities. The software was a Microsoft ACCESS database that could be used to store the VFC/AFIX site visit data in accordance with the report requirements due at the end of each calendar year. The VFC/AFIX Evaluation Software was initially distributed to the grantees in 2002. In 2004, this software was redesigned to be incorporated as a module in CoCASA; and in January 2005 it was released as a stand alone software application prior to the completion of the full CoCASA. Once the entire CoCASA was completed and released in January 2006, data entered into the stand alone evaluation module was transferred to the full CoCASA. CoCASA could then be used to tabulate the data for the 2005 VFC Management Survey. Grantees are not required to use the VFC/AFIX Evaluation module of CoCASA; however, they are advised to develop their own tracking system that would capture the same data fields if they choose not to use the software. VFC/AFIX Staff The number of full-time employees (FTEs) working on VFC/AFIX activities is tabulated in Table 1 below. In total, 275 FTEs are employed at the state level, and 259 FTEs are employed at the local level. Together, 534 FTEs at the state or local level are currently working on VFC/AFIX related activities across the country. Additionally, there are 98 contract staff members working on VFC/AFIX related activities at the state or local level. As shown in Table 1, these numbers can be categorized by new and existing positions for calendar year 2005. The grantee VFC/AFIX projects can be carried out at the state level or the local level by immunization program staff or by staff hired through contracts with outside agencies. Table 1. Number of FTEs Working on VFC/AFIX Project CY2005 Personnel Program Staff* Contract Staff** Total State Local State Local State Local New Positions 12 6 3 0.0 15 6 Existing Positions 263 253 60 35 323 288 Total 275 259 63 35 338 294 *State Program Staff: state-employed staff working on VFC/AFIX at the state immunization program level. Local Program Staff: local health department staff funded with federal funds who work on VFC/AFIX. **State Contract Staff: VFC/AFIX staff hired by the state immunization programs through third party contracts with federal funds. Local Contract Staff: VFC/AFIX staff at local health departments hired through contracts with outside agencies using federal funds. Provider Information Table 2 below includes the number of provider sites that received at least one visit during 2005 for each category of provider. Public providers are divided into three categories. The “Public” category includes local health departments and Indian Health Service clinics. Community or Migrant Health Centers (C/MHC) includes Federally Qualified Health Care Centers, and the “Other Public” category captures all other facilities that are not included in the other two categories. “Private” represents all private providers that received at least one visit in 2005. The providers are categorized into “VFC-enrolled sites” and “Non-VFC enrolled sites.” Including both types of provider sites, a total of 23,200 provider sites received at least one visit in calendar year 2005. Of the 23,200 provider sites visited, 99% were enrolled in the VFC program. The 22,910 VFC-enrolled provider sites that were visited constitute 51% of the total active VFC-enrolled provider sites, a decrease from 55% (24,400 of 44,101 provider sites) in 2004. Table 3 illustrates the percentage provider sites visited in the public sector and the private sector. The percentage of provider sites visited in the private sector decreased from 53% in 2004 to 48% in 2005. Table 2. Number of Provider Sites Receiving at Least One visit, CY2005 Provider Information Public C/MHC* Other Public Subtotal All Public Private Total VFC-enrolled Provider Sites 3515 2594 1145 7254 15,656 22,910 Non-VFC Provider Sites 17 10 25 52 244 296 Total 3520 2605 1172 7297 15,903 23,200 *Community or Migrant Health Center Table 3. Number and Percent of VFC-enrolled Provider Sites Receiving at Least One Visit, CY2005 Provider Information All Public Private Total Number of VFC-enrolled Provider Sites 12,019 32,506 44,525 Number (Percent) of VFC-enrolled Providers Who Received at Least One Visit, CY2004 7254 (60%) 15,656 (48%) 22,910 (51%) Site Visit Information While 22,910 VFC-enrolled provider sites were visited in 2005, the actual number of VFC-enrollment visits, VFC Only, AFIX Only, VFC/AFIX combined, and educational visits to a VFC-enrolled provider site totaled 29,809. Table 4 below details the number of visits to VFC and Non-VFC provider sites by visit type and provider type. As the table shows, 9339 visits were conducted in public VFC-enrolled sites (public + c/mhc + other public) and 20,225 visits were conducted in private VFC-enrolled sites. In addition, 245 visits (AFIX Only and Educational) were conducted in Non-VFC enrolled provider sites (public non-vfc + c/mhc non-vfc + other public non-vfc + private non-vfc). Table 4. Total Number of Visits by Provider Type, CY2005 Type of Visit Public VFC Enrolled Public Non-VFC C/MHC VFC Enrolled C/MHC Non-VFC Other Public VFC Enrolled Other Public Non-VFC Private VFC Enrolled Private Non-VFC VFC Enrollment 25 151 82 1190 VFC Only* 1201 910 489 4579 AFIX Only** 818 1 245 4 30 0 937 206 VFC/AFIX Combined*** 2017 1464 568 9499 Educational 663 6 513 0 163 13 4020 15 Total 4,724 7 3,283 4 1,332 13 20,225 221 * VFC Only is defined as a visit to a VFC enrolled provider to ensure compliance with VFC program requirements. ** AFIX Only is defined as a quality improvement strategy utilizing assessment of immunization records, feedback, incentive, and exchange of information through performance measurement, diagnosis of service delivery problems, and data feedback during a visit to a medical practice. One AFIX visit should contain an assessment and a feedback component even though more than one physical visit to the provider site may be required to complete the assessment and the feedback sessions. *** A VFC/AFIX Combined site visit is defined as a visit to a VFC-enrolled provider site which integrates the review to ensure compliance with VFC program requirements and immunization record assessment and feedback activities. Table 5 includes additional documentation regarding site visits in CY 2005. Repeat AFIX visits are a subset of the AFIX visits documented in Table 4, and they are included to illustrate the number of visits that occurred at a site that had previously received an AFIX visit between January 1, 2000, and December 31, 2004. The nature of AFIX as a continuous quality improvement strategy requires that provider sites are visited on more than one occasion to evaluate incremental progress. The number of repeat AFIX visits allows CDC/NIP to track the grantees’ progress in implementing this ongoing quality improvement strategy. The VFC Follow-Up visits describe the number of visits that occurred as a result of a problem and/or concern found during a VFC visit. The information in Tables 4 and 5 reveals that grantees are actively visiting provider sites, following up on problems identified in previous visits, providing education as well as service, and ultimately building relationships with staff. Table 5. Additional Visits, CY2005 Type of Visit Public VFC Enrolled Public Non-VFC C/MHC VFC Enrolled C/MHC Non-VFC Other Public VFC Enrolled Other Public Non-VFC Private VFC Enrolled Private Non-VFC Repeat AFIX* 1161 5 592 1 205 4 3086 99 Follow-up VFC** 1252 422 303 2710 *Repeat AFIX: the number of AFIX visits (includes AFIX only and Combined VFC/AFIX) from Table 4 that are repeat assessments (e.g. the provider received an assessment during a previous year). **Follow-up VFC: the number of visits completed to evaluate provider response to previously identified problems found during the initial VFC site visit. As part of the annual grant application process, grantees are required to specify the proposed number of site visits to be conducted in the upcoming calendar year. For the 2005 grant applications, the planned number of site visits included three categories: VFC Only, AFIX Only and VFC/AFIX Combined. In an effort to examine the accuracy with which a grantee can estimate VFC/AFIX activities, the number of proposed site visits from the grant applications were compared with the actual number of site visits (for the categories of VFC Only, AFIX Only and VFC/AFIX Combined). Grantees exceeded the total number of proposed site visits in CY 2005 (see Table 6). Table 6. Proposed and Actual Number of Site Visits, CY2005 Type of Site Visit Public Provider* Private Provider Total Visits Proposed Actual Proposed Actual Proposed Actual VFC only 2025 2600 4551 4579 6576 7179 AFIX only 931 1098 1189 1143 2120 2241 VFC/AFIX combined 5290 4049 10,295 9499 15,585 13,548 Total visits 7807 7747 15,024 15,221 22,831 22,968 *Public provider includes Public, C/MHC, and Other Public. Assessment Outcome Measures for Public Providers Of the 61 grantees that receive funds for VFC-AFIX activities, 61 submitted public provider data for the annual report (see Appendix B, Table 1). Of the 61 that submitted data, 55 use the Standard assessment method (designated as “Standard”). No grantees reported only using the Hybrid assessment method; six grantees use a combination of the Standard and Hybrid assessment methods (designated as “Both”). Information on the difference between the Standard and Hybrid assessment methods is found in Appendix C. Grantees that use the Standard assessment method will receive an estimated immunization coverage level for each provider assessed. Grantees that use the Hybrid assessment method will receive a result indicating the provider is “above” or “below” a pre-determined threshold immunization coverage level (see Appendix C for more in-depth explanation). The majority of the grantees (32/60) assess children 24-35 months of age. Other age groups reported include 19-35 months (14 grantees) and 12-23 months (1grantee). Six grantees reported that they assess both ages 12-23 and 24-35 months. Seven grantees responded to this question with “other.” One grantee did not report this information. From the data submitted by 61 grantees, 4401 public provider sites were assessed using the Standard assessment method. Of these 4401 public provider assessments, 1911 (43%) of the assessments estimated vaccination coverage levels greater than 80%. An additional 658 (15%) of the assessments estimated vaccination coverage levels between 70% and 79%; the remaining assessments estimated vaccination coverage less than 70%. The assessment results recorded during calendar year 2005 represent the baseline measure for comparing vaccination level improvement in this manner. In previous years, a crude average of all estimated coverage levels was reported by the grantees; however, this new method of documenting the number of assessments within an estimated vaccination coverage level range will better reveal improvements than a crude average of the vaccine coverage level estimates. It is anticipated that over time, the number of providers with estimated vaccination levels greater than 80% will increase and the number of providers with estimated vaccination levels less than 70% will decrease. For those grantees using the Hybrid assessment method, threshold levels of 70, 75, and 80 percent were used. For those grantees that used the Hybrid assessment method, 3 of 5 grantees reported the number of providers above the selected threshold level as greater than 60% of the providers assessed. Note: The above numbers are not mutually exclusive – meaning one grantee could have done some AFIX visits using the Hybrid assessment method and some visits using the Standard assessment method. Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5 Assessment Outcome Measures for Private Providers Of the 61 grantees that receive funds for VFC-AFIX activities, 61 submitted private provider data for the annual report (see Appendix B, Table 2). Of the 61 that submitted data, 55 use the Standard assessment method (designated as “Standard”). No grantees use only the Hybrid assessment method; 6 grantees use a combination of the Standard and Hybrid assessment methods (designated as “Both”). The majority of the grantees (32/60) assess children 24-35 months of age. Other age groups reported include 19-35 months (14 grantees) and 12-23 months (1 grantees); and 6 grantees reported that they assess children ages 12-23 and 24-35 months of age. Seven grantees responded to this question with “other.” One grantee did not report this information. From the data submitted by 61 grantees, 8444 private provider sites were assessed using the Standard assessment method. Of these 8444 private provider assessments, 3728 (44%) of the assessments estimated vaccination coverage levels greater than 80%. An additional 1250 (15%) of the assessments estimated vaccination coverage levels between 70% and 79%; the remaining assessments estimated vaccination coverage less than 70%. The assessment results recorded during calendar year 2005 represent the baseline measure for comparing vaccination level improvement in this manner. In previous years, a crude average of all estimated coverage levels was reported by the grantees; however, this new method of documenting the number of assessments within an estimated vaccination coverage level range will better reveal improvements than a crude average of the vaccine coverage level estimates. It is anticipated that over time, the number of providers with estimated vaccination levels greater than 80% will increase and the number of providers with estimated vaccination levels less than 70% will decrease. For those providers using the Hybrid assessment method, threshold levels of 70, 75, and 80 percent were used. All of the grantees using the Hybrid Method (5 of 5) reported 70% or more of the providers assessed as above the designated threshold level. Note: The above numbers are not mutually exclusive – meaning one grantee could have done some AFIX visits using the Hybrid assessment method and some visits using the Standard assessment method. Note: Coverage levels were rounded to the nearest whole percentage point for decimals ?0.5 Limitations The data submitted by grantees through the VFC Management Survey were reviewed on an individual basis and if data were missing or seemed inappropriate (i.e. the number of providers that received a visit was more than the number of visits made), follow-up calls were made. During this data cleaning process, several limitations were discovered. The data submitted for calendar year 2005 was the first time the new software product, CoCASA, was used for documentation of VFC and AFIX activities. The data requested in the VFC Management Survey could be generated using the report functions in CoCASA; however, these aggregate reports were dependent upon the manner in which grantees described and/or categorized their activities and the amount of information that was documented by the grantee. There was some confusion over how to enter the data and/or how to describe certain activities. Some grantees did not have processes in place to document the requested information on a daily basis and therefore the report results were not as robust as they could have been. For some grantees, data were entered at the end of the year so recalling the results of a visit that took place 10 months prior was not possible. The outcome measures for each assessment visit were not always documented. The reason could have been a technical problem (i.e. lost data) or as suggested above, processes were not in place to routinely document this information. This caused a mismatch between the number of assessment visits documented and the available assessment results. Every attempt was made to clarify and clean the data; however, due to the timeframe from submission of data and the due date of this report all discrepancies in the data could not be resolved in a timely manner.  To minimize the risk of this occurring again in 2007, CDC will provide technical assistance and education through individual conference calls, group conference calls (our quarterly conference calls), and emails. The content of these communications will focus on instructing the grantees on how to properly collect and document VFC and AFIX activities. Summary of VFC-AFIX Activities for 2005 The VFC-AFIX Activities in 2005 included two major milestones: the formal release of the AFIX Standards, Level I and the availability of the VFC-AFIX Evaluation Module of CoCASA. The AFIX Standards are designed to assist grantees in the formalization of AFIX processes that will produce more quality improvement focused activities and outcomes. The VFC-AFIX Evaluation Module of CoCASA is also designed to assist grantees, particularly in expediting the creation of reports that summarize the VFC-AFIX activities and outcomes from the previous year. The data are self-reported by the grantee and NIP staff has realized through the data cleaning process that more education and technical assistance must be provided to the grantees throughout 2006 as they document their VFC and AFIX activities (and outcomes). The primary focus will be to consistently and methodically categorize activities and document them using standardized protocols. The AFIX Standards Level I should assist the grantees in the development of these standard protocols; NIP staff’s responsibility will be to ensure they are implemented appropriately. In the following year, it is anticipated that increased familiarity with the CoCASA software as well as increased technical assistance from NIP will improve the quality of the data. Reporting VFC-AFIX Activities for 2006 During 2006, CDC proposed to merge Quality Assurance Reviews (QARs), provider site visits statutorily required by the VFC program, into the VFC-AFIX program and to merge operations funds currently supporting these visits into the VFC-AFIX budget line. To reflect this merger, a summary of all quality assurance activities will be included in this annual report beginning in the summary for CY 2006. This report currently includes information on the total number of QAR visits performed by grantees and those QAR visits are referred to as “VFC only” visits within the report. Beginning next year, CDC will also include information about the operations funding awarded to support this activity (Appendix 1). APPENDIX A VFC-AFIX Funding Grantee 2005 VFC AFIX Award Alabama $213,000 Alaska $112,000 American Samoa $8,762 Arizona $494,000 Arkansas $226,000 California $1,417,000 Chicago $291,000 Colorado $205,000 Connecticut $387,000 Delaware $174,000 District of Columbia $52,000 Florida $503,000 Georgia $243,000 Guam $46,829 Hawaii $315,000 Houston $339,000 Idaho $171,000 Illinois $312,000 Indiana $389,000 Iowa $117,000 Kansas $115,000 Kentucky $157,000 Louisiana $112,000 Maine $139,000 Maryland $82,000 Massachusetts $348,000 Michigan $321,000 Minnesota $137,000 Mississippi $222,000 Missouri $343,000 Montana $49,000 Nebraska $112,000 Nevada $365,000 New Hampshire $123,000 New Jersey $257,000 New Mexico $192,000 New York City $184,000 New York State $625,000 North Carolina $132,000 North Dakota $35,000 Ohio $193,000 Oklahoma $209,000 Oregon $175,000 Pennsylvania $639,000 Philadelphia $197,000 Puerto Rico $443,000 Rhode Island $110,283 San Antonio $92,000 South Carolina $563,000 South Dakota $90,000 Tennessee $289,000 Texas $2,117,000 Utah $278,000 Vermont $103,000 Virgin Islands $93,533 Virginia $146,000 Washington $348,917 West Virginia $65,000 Wisconsin $200,000 Wyoming $82,000 Total $ 16,499,324 APPENDIX B Assessment Outcomes Appendix B, Table 1: Assessment Outcome Measures for Public Providers by Grantee, CY2005 Grantee Age Group Assessment Vaccination Number Number of Number of Number of Vaccination Number Hybrid Number Of Assessed Method Series assessed providers providers providers Series Assessed Threshold Providers (Months) Assessed with with with with Assessed With Level At or Above with Standard Standard coverage coverage coverage With Hybrid Hybrid Threshold Method Method 0-69% 70-79% 80-100% Method Method Level Alabama 24-35 standard 431331 29 13 2 14 Alaska 19-35 standard 43133 27 8 8 11 American Samoa 19-35 both 43133 6 1 5 Arizona 24-35 standard 43133 24 10 4 Arkansas 24-35 standard 43133 92 7 11 74 California Other both 43133 296 126 57 113 43133 3 80 2 Chicago 12 & 24 both 43133 19 9 4 6 43133 29 75 24 Colorado 19-35 both 431331 36 14 7 15 431331 10 70 3 Connecticut 19-35 standard 431331 14 3 3 8 Delaware 19-35 standard 431331 0 District of Columbia 19-35 standard 43133 19 5 9 5 Florida 24-35 standard 431331 67 12 6 49 Georgia 24-35 standard 431331 199 71 14 114 Guam 24-35 standard 43133 2 2 0 0 Hawaii 24-35 standard 43133 11 3 3 5 Houston 24-35 standard 431331 0 Idaho 19-35 standard 43133 32 8 4 20 Illinois 12 & 24 standard 43133 134 71 20 43 Indiana 19-35 standard 431331 166 65 25 76 Iowa 24-35 standard 431331 142 30 25 87 Appendix B, Table 1: Assessment Outcome Measures for Public Providers by Grantee, CY2005 Grantee Age Group Assessment Vaccination Number Number of Number of Number of Vaccination Number Hybrid Number Of Assessed Method Series assessed providers providers providers Series Assessed Threshold Providers (Months) Assessed with with with with Assessed With Level At or Above with Standard Standard coverage coverage coverage With Hybrid Hybrid Threshold Method Method 0-69% 70-79% 80-100% Method Method Level Kansas 12 & 24 standard 43133 129 26 13 90 Kentucky 24-35 standard 431331 128 15 13 100 Louisiana 19-35 standard 431 99 41 30 28 Maine 19-35 standard 43133 56 38 5 13 Marianas Islands standard 43133 Maryland 24-35 standard 431331 29 3 7 19 Massachusetts 24-35 both 43133 13 3 4 6 43133 24 80 10 Michigan 19-35 both 431331 45 17 11 17 431331 89 70 55 Minnesota 24-35 standard 431331 32 29 0 3 Mississippi 24-35 standard 431 145 22 22 101 Missouri 12 & 24 standard 431331 249 174 34 41 Montana 24-35 standard 43133 97 8 5 84 Nebraska 24-35 standard 431331 25 12 4 9 Nevada 24-35 standard 43133 24 24 0 0 New Hampshire Other standard 43133 9 2 1 6 New Jersey 24-35 standard 431331 49 12 11 26 New Mexico Other standard 431331 71 29 10 32 New York City 24-35 standard 431331 20 8 7 5 New York State 24-35 standard 43133 66 27 8 31 North Carolina 24-35 standard 43133 131 23 15 93 Appendix B, Table 1: Assessment Outcome Measures for Public Providers by Grantee, CY2005 Grantee Age Group Assessment Vaccination Number Number of Number of Number of Vaccination Number Hybrid Number Of Assessed Method Series assessed providers providers providers Series Assessed Threshold Providers (Months) Assessed with with with with Assessed With Level At or Above with Standard Standard coverage coverage coverage With Hybrid Hybrid Threshold Method Method 0-69% 70-79% 80-100% Method Method Level North Dakota 12 & 24 standard 43133 21 3 4 14 Ohio 24-35 standard 431331 138 Oklahoma 24-35 standard 431331 123 81 22 20 Oregon 12 & 24 standard 43133 80 57 18 5 Pennsylvania 24-35 standard 431331 65 12 15 38 Philadelphia 19-35 standard 43133 13 3 4 6 Puerto Rico Other standard 431331 209 143 27 39 Rhode Island 12-23 standard 3222 16 1 3 12 San Antonio Other standard 431331 25 10 1 14 South Carolina 24-35 standard 43133 68 30 12 26 South Dakota 19-35 standard 43133 122 14 7 101 Tennessee 24-35 standard 431331 73 28 15 30 Texas Other standard 431331 489 219 97 173 Utah 24-35 standard 431331 32 20 5 7 Vermont 24-35 standard 43133 23 14 3 6 Virgin Islands 24-35 standard 431331 3 2 1 0 Virginia 24-35 standard 431331 16 6 4 6 Washington 19-35 standard 43133 36 26 0 10 West Virginia 24-35 standard 43133 58 22 5 31 Wisconsin Other standard 431331 18 7 5 6 Wyoming 24-35 standard 43133 41 16 7 18 Appendix B, Table 2: Assessment Outcome Measures for Private Providers by Grantee, CY2005 Grantee Age Group Assessment Vaccination Number Number of Number of Number of Vaccination Number Hybrid Number Of Assessed Method Series assessed providers providers providers Series Assessed Threshold Providers At (Months) Measured with with with with Assessed With Hybrid Level or Above With Standard Standard coverage coverage coverage With Hybrid Threshold Method Method 0-69% 70-79% 80-100% Level Alabama 24-35 standard 431331 77 28 19 30 Alaska 19-35 standard 43133 13 3 4 6 American Samoa 19-35 both 43133 0 Arizona 24-35 standard 43133 174 55 35 84 Arkansas 24-35 standard 43133 103 57 14 32 California Other both 43133 89 46 21 22 43133 140 80 106 Chicago 12 & 24 both 43133 114 71 14 29 43133 62 75 48 Colorado 19-35 both 431331 0 0 0 0 431331 68 70 52 Connecticut 19-35 standard 431331 87 21 7 59 Delaware 19-35 standard 431331 11 2 6 District of Columbia 19-35 standard 43133 43 31 9 3 Florida 24-35 standard 431331 320 81 59 180 Georgia 24-35 standard 431331 233 48 22 163 Guam 24-35 standard 43133 8 7 1 0 Hawaii 24-35 standard 43133 47 16 4 27 Houston 24-35 standard 431331 227 108 46 73 Idaho 19-35 standard 43133 111 51 13 47 Illinois 12 & 24 standard 43133 210 66 31 113 Indiana 19-35 standard 431331 368 147 61 160 Iowa 24-35 standard 431331 73 34 13 26 Appendix B, Table 2: Assessment Outcome Measures for Private Providers by Grantee, CY2005 Grantee Age Group Assessment Vaccination Number Number of Number of Number of Vaccination Number Hybrid Number Of Assessed Method Series assessed providers providers providers Series Assessed Threshold Providers At (Months) Measured with with with with Assessed With Hybrid Level or Above With Standard Standard coverage coverage coverage With Hybrid Threshold Method Method 0-69% 70-79% 80-100% Level Kansas 12 & 24 standard 43133 42 17 5 20 Kentucky 24-35 standard 431331 42 17 2 23 Louisiana 19-35 standard 431 87 39 10 38 Maine 19-35 standard 43133 158 89 26 43 Marianas Islands standard 43133 0 Maryland 24-35 standard 431331 516 65 46 405 Massachusetts 24-35 both 43133 130 27 16 87 43133 147 80 122 Michigan 19-35 both 431331 142 55 35 52 431331 327 70 235 Minnesota 24-35 standard 431331 61 53 6 2 Mississippi 24-35 standard 431 95 15 16 64 Missouri 12 & 24 standard 431331 201 113 37 51 Montana 24-35 standard 43133 60 1 2 57 Nebraska 24-35 standard 431331 126 42 17 67 Nevada 24-35 standard 43133 52 41 8 3 New Hampshire Other standard 43133 60 10 5 45 New Jersey 24-35 standard 431331 159 64 30 65 New Mexico Other standard 431331 101 43 18 40 New York City 24-35 standard 431331 105 72 15 18 New York State 24-35 standard 43133 227 65 41 121 North Carolina 24-35 standard 43133 85 29 18 38 Appendix B, Table 2: Assessment Outcome Measures for Private Providers by Grantee, CY2005 Grantee Age Group Assessment Vaccination Number Number of Number of Number of Vaccination Number Hybrid Number Of Assessed Method Series assessed providers providers providers Series Assessed Threshold Providers At (Months) Measured with with with with Assessed With Hybrid Level or Above With Standard Standard coverage coverage coverage With Hybrid Threshold Method Method 0-69% 70-79% 80-100% Level North Dakota 12 & 24 standard 43133 16 0 5 11 Ohio 24-35 standard 431331 300 Oklahoma 24-35 standard 431331 173 146 6 21 Oregon 12 & 24 standard 43133 65 42 13 10 Pennsylvania 24-35 standard 431331 432 167 72 193 Philadelphia 19-35 standard 43133 87 37 18 32 Puerto Rico Other standard 431331 0 0 0 0 Rhode Island 12-23 standard 3222 50 4 1 45 San Antonio Other standard 431331 95 21 7 67 South Carolina 24-35 standard 43133 38 12 7 19 South Dakota 19-35 standard 43133 90 9 9 72 Tennessee 24-35 standard 431331 86 33 26 27 Texas Other standard 431331 1744 773 267 704 Utah 24-35 standard 431331 56 13 8 35 Vermont 24-35 standard 43133 42 17 8 17 Virgin Islands 24-35 standard 431331 6 5 0 1 Virginia 24-35 standard 431331 164 46 30 88 Washington 19-35 standard 43133 107 63 18 26 West Virginia 24-35 standard 43133 30 6 7 17 Wisconsin Other standard 431331 46 27 6 13 Wyoming 24-35 standard 43133 60 15 14 31 APPENDIX C Assessment Methods Method Description Advantages Disadvantages Chart Based: Standard Assessment OPTION #1: Number of Charts: all eligible records within a specified age group* Method for Selecting Charts: select all records within the specified age range Software to Use: CASA Information Entered into Software: child’s demographic information; date of each immunization; other information related to diagnostic analysis Software generated outcomes: immunization coverage level, diagnostic information on late starts, drop-offs, missed opportunities, etc * Precise estimates of immunization coverage levels * Evaluation of missed opportunities * Evaluation of late starts, etc. * Can be a resource burden (staff, time) if the entire group of children in the specified cohort is large. Chart Based: Standard Assessment OPTION #2: Number of Charts: minimum of 50 medical charts selected in specified age group that will be assessed on the same immunizations (i.e. 50 children 12-23, 19-35 or 24-35 rather than selecting 50 charts for children 12-35 months of age.) Method for Selecting Charts: random sample, systematic random sample, shelf method or convenience sample Software to Use: CASA Information Entered into Software: child’s demographic information; date of each immunization; other information related to diagnostic analysis Software generated outcomes: immunization coverage level, diagnostic information on late starts, drop-offs, missed opportunities, etc * Use of sample reduces time pulling charts and entering data. * Coverage level produced is an estimate. * The sample may not be randomly selected. Chart Based: Hybrid Assessment Number of Charts: 30 charts for children 19-35 months of age only Method for Selecting Charts: random sample, systematic sample, shelf method, or convenience Software to Use: VFC-CASA (NOT CASA) Information Entered into Software: selected demographic information, number of doses for each Immunization OR dates for each immunization Software generated outcomes: whether a provider has immunization coverage above or below a specified threshold and option to produce immunization histories of not up-to-date clients as examples to discuss with the provider and staff * Smallest sample size * Rapid assessment * Identifies providers who may benefit from a standard and more thorough assessment. * Does not give point estimate of coverage. * Smaller basis for diagnostic feedback – can only provide case by case examples. Registry Based Assessment Number of charts: all eligible records within the pre-determined age group Method for selecting charts: select all records within the specified age range Software to use: CASA Information entered into software: demographic and immunization data selected from registry Software generated outcomes: immunization coverage level, diagnostic information on late starts, drop-offs, missed opportunities, etc * Minimal time and effort for data collection. * No sampling error since estimates based on census of records. * Potential bias if registry does not include all of the provider’s records * Potential bias if provider database does not include all historical records * Reliability of registry data Attachment A Vaccine University Student Handbook Attachment B Vaccine University Evaluation Summary Attachment C National Immunization Program Annual Report 1