A Progress Report on the VFC-AFIX Initiative:
 A National Strategy to Improve the Quality of 
Immunization Practices among VFC Providers
May 2003 

Background/Introduction
The year 2002 represented the second year that 61 of the 64 eligible grantees that requested funding for the Vaccines for Children (VFC) –Assessment Feedback Incentives and eXchange (AFIX) initiative received funding for program planning and implementation.  This initiative is to assure that VFC-eligible children are receiving quality services and to assure program compliance

In 1995, the Congress directed the Centers for Disease Control and Prevention (CDC) to set guidelines for grantees receiving federal funds for immunization programs to assess coverage levels in all public clinics. A continuous quality improvement strategy was developed to assess coverage levels and provide insight on how to improve coverage levels.  This strategy is known by the acronym of “AFIX”.  The four components of “AFIX” are Assessment of immunization coverage level, Feedback of information to providers, Incentives to institute change and eXchange of information on best practices.  This strategy has been documented in the CDC Guide to Community Preventive Services as an effective method to improve and sustain immunization coverage levels.

The VFC program is widely accepted by private health care providers that serve VFC eligible children.  Private provider participation has created a “shift” in the provision of vaccination services over the last 10 years from public health clinics to private health care offices.  Recognizing this shift, an objective related to assessment of immunization coverage level was included in the Healthy People 2010 goals. The objective is to “increase the proportion of providers who have measured the vaccination coverage levels among children in their practice population within the last two years.”  The Healthy People 2010 report included a baseline measurement for private providers as 6% and a baseline measurement for public providers as 66%.  These baseline measurements were established in 1997. 

The purpose of this report is to provide a review and analysis of programmatic activities at the state and federal level that occurred in the second year that all eligible grantees participated in the VFC-AFIX project.  The report also will summarize the findings from the information submitted by the grantees for the year 2002 and discuss programmatic challenges that were faced in 2002 at grantee and federal levels.  

2002 Administrative Activities
Three significant changes occurred within the VFC/AFIX project at the National Immunization Program (NIP) in 2002.  The first was the creation of a Clinical Assessment Software Application (CASA) help desk position within the Program Support Branch (PSB) of the Immunization Services Division (ISD).  Three major responsibilities of this position are to be the point of contact for CASA related 
questions/problems for the grantees, provide oversight and maintenance to the AFIX website (www.cdc.gov/nip/afix), and provide training and education on CASA as requested by the grantees.   This position was filled in January 2002.

The second change was a reorganization of the leadership of the VFC/AFIX project within PSB.  The VFC/AFIX project now has rotating leadership.  The leadership of the project rotates between the three staff assigned to this project within PSB. The rotation will occur annually on August 1st.  This has allowed the different PSB staff assigned to this project to expand their knowledge and skill on different aspects of this project.

The third change focused on improving how PSB staff provide service and technical assistance to the grantees. In response to the President’s Management Agenda, an evaluation plan was developed to assess individual grantee’s progress and needs in implementing and evaluating the VFC/AFIX initiative.  The evaluation plan is based on data submitted to ISD/PSB annually in VFC/AFIX evaluation software, narrative information presented in the grant application and information obtained from standardized questions asked during site visits reviews conducted by Program Operations Branch (POB) consultants.  The VFC/AFIX staff at NIP will review all three data sources for each grantee using a standardized tool develop by the PSB staff. Based on the findings, the grantees will be placed into one of four categories ranging from outstanding to poor. The objective of this activity is to extend an offer of individual technical assistance to 20% of the lowest performing grantees within twelve months of identification.  Since several of the data sources used for this evaluation have staggered CDC submission dates ranging from fall 2002 to spring 2003, the implementation of this plan will begin in the summer of 2003. 

2002 NIP Programmatic Activities
In addition to the aforementioned changes, many other activities enhanced the VFC/AFIX project at the federal, state and local level. This section of the report outlines some of the key 2002 activities:
•	VFC/AFIX project staff revised the AFIX section in the “Epidemiology and Prevention of Vaccine-Preventable Diseases” manual in March 2002.
•	VFC/AFIX project staff finalized and published the “Core Elements for AFIX Training and Implementation”.
•	VFC/AFIX project staff revised the AFIX website.
•	VFC/AFIX project staff in conjunction with grantee staff presented the “Core Elements for AFIX Training and Implementation” at the National Immunization Conference (NIC) in April 2002.
•	VFC/AFIX project staff coordinated and/or moderated other sessions at the 2002 NIC on related VFC/AFIX workshops including, “The “I” of AFIX: Incentives, Recognition and Rewards”, “The AFIX Exchange”,  “ A Beginner’s Guide to Successful VFC/AFIX Activities”, and “Methods for Performing Registry Based Coverage Assessments”.
•	VFC/AFIX staff developed a scope of work and contracted with an outside vendor to conduct usability testing and create a programming blueprint to allow NIP to redesign the Clinical Assessment Software Application (CASA) into a more “user-friendly” format. The contract period between NIP and the vendor is October 2002-April 2003.
•	VFC/AFIX staff coordinated quarterly grantee conference calls to facilitate discussion between grantees and NIP on VFC/AFIX activities.
•	VFC/AFIX staff reviewed nominations and presented the AFIX award at the 2002 NIC to the Indiana Immunization program.
•	VFC/AFIX staff convened a working group consisting of grantee representatives and NIP staff to revise and pilot test a new VFC Site Visit Questionnaire.   The revised VFC Site Visit Questionnaire will be implemented by all grantees in 2003.
•	VFC/AFIX staff began development of standardized procedures for monitoring and responding to cold chain storage issues identified in VFC-enrolled provider sites.
•	VFC/AFIX staff revised the VFC/AFIX evaluation software for use in 2003.

2002 Grantee Programmatic Activities
Grantees actively improved VFC/AFIX operations.  This section of the report illustrates some of the VFC/AFIX activities conducted by the grantees in 2002: 
•	Ohio developed an AFIX orientation/training manual for new employees in March 2002.
•	Utah implemented an awards program to recognize high performers and most improved practices through implementing best practices.
•	Utah Immunization Program conducted a “Best Practices” Workshop in order for providers to exchange ideas.
•	North Carolina used graphs without identifiers to compare provider coverage rates so that each provider could compare his/her own rates with other provider rates in the state.
•	Rhode Island conducted a missed opportunities study and began work on a “Best Practices” manual.
•	New Mexico a statewide Immunizations Awards Dinner was held in 2002 to honor those providers with the most improved coverage and also with the highest coverage.  High level involvement by the Secretary of Health and Children’s Cabinet members was obtained.
•	Louisiana increased by 69% the statewide on-site VFC/AFIX Active Private Provider site visits over the previous CY 2001.
•	Pennsylvania (PA) continues its collaborative efforts with the PA chapter of the American Academy of Pediatrics (AAP) to promote the Immunization Education program (IEP).  The IEP is a peer to peer model providing outreach, education, and assessment activities to the medical community. The IEP teams, consisting of a physician, office practice manager, and public health nurse, conducted 61 CASA assessments and 108 presentations to 2,226 participants in CY2002.
•	San Antonio evaluated 102 providers using the 4 DTaP-3 Polio-1 MMR-3 Hib-3 HBV series.  A 12% increase in coverage was observed compared to 2001 coverage assessment data for the same providers.
•	San Antonio also publishes “Shot Talk”. In each issue of the publication individual provider practices that achieved or exceeded 80% immunization coverage rates were highlighted. Twenty-three VFC providers were recognized for coverage levels at or above 80%.
•	Wisconsin presented results from the provider CASA immunization assessments without personal identifiers to the Wisconsin Council on Immunization Practices (WCIP).  The data were well received, and the WCIP members agreed that further distribution of this data should be listed without personal identifiers so as to promote good practice with out embarrassing any provider.  
•	Of the 1248 visits completed in California, 48% had a provider present when feedback was presented, while 65% of providers were available to answer questions regarding the 18 standards for pediatric practices.  Improvement was observed across all Quality Assurance Review results.  Significant improvement has been made in the measurement of factors related to improving immunization coverage.  For example, 60% of providers who in the past deferred immunizations if a child had a mild illness are now vaccinating. 
•	In 2002, 589 VFC and VFC/AFIX visits were conducted in public and private VFC provider sites in New York State. Vaccine handling procedures and compliance with eligibility requirements were assessed at each VFC visit. The AFIX collaboration with county staff continued in 2002, and as a result, 181 AFIX-only site visits were completed. The average rate for 4-3-1-3-3 was 70 percent in 2002, an increase of three percent from the previous year. 

2002 Grantee Submitted Data Analysis
CDC/NIP created the VFC/AFIX Evaluation Software as a tool for grantees to monitor their VFC/AFIX activities. This software is a MS ACCESS database that can be used to store the VFC/AFIX site visit data in accordance with the report requirements due at the end of each calendar year. If this software is used by the grantees, then the aggregate data required for submittal to CDC/NIP can be automatically generated by the report options built into the software. The VFC/AFIX Evaluation Software was distributed to the grantees in 2002. 

Grantees are not required to use the VFC/AFIX Evaluation Software; however, they were advised to develop their own tracking system that would capture the same data fields if they chose not to use the software. Grantees were required to submit documentation of their 2002 VFC/AFIX activities as part of the “Annual VFC Management Survey” due March 1st, 2003. The “Annual VFC Management Survey” data were collected using a web-based reporting method. Of the 61 eligible grantees, 59 submitted data for the “Annual VFC Management Survey”.  Data from American Samoa and Guam were not received. 

VFC/AFIX Staff
The number of full time employees (FTEs) working on VFC/AFIX activities is tabulated in Table 1 below. In total, 258 FTEs are employed at the state level and 181 FTEs are employed at the local level. Together, 439 FTEs are currently working on VFC/AFIX related activities across the country. 

As shown in Table 1, these numbers can be categorized by new and existing positions for calendar year 2002 and the grantee VFC/AFIX projects can be carried out at the state level or the local level by immunization program staff or by staff hired through contracts with outside agencies. 

Table 1. Number of FTEs Working on VFC/AFIX Project CY2002
Personnel	Program Staff*	Contract Staff**	Total	
	State	Local	State	Local	State	Local	
New Positions	24	10	13	0	37	10	
Existing Positions 	172	144	49	28	221	172	
Total 	196	154	62	28	258	181	
*State Program Staff: state employed staff working on VFC/AFIX at the state immunization program level. Local Program Staff: local health department staff funded with federal funds that work on VFC/AFIX.
**State Contract Staff: VFC/AFIX staff hired by the state immunization programs through third party contracts with federal funds. Local Contract Staff: VFC/AFIX staff at local health departments hired through contracts with outside agencies using federal funds. 

Provider Information
Table 2 below includes the number of provider sites that received at least 1 visit during 2002 for each category of provider. Public providers include local health departments and Indian Health Service clinics while “Other Public” represents agencies that are not Community or Migrant Health Centers (C/MHC) and are not included in the “Public” category. “Private” represents all private providers that received at least 1 visit in 2002. The providers are categorized into “VFC-Enrolled sites” and “Non-VFC Enrolled sites”. Including both types of provider sites, a total of 20,836 provider sites received at least one visit in calendar year 2002. Of the 20,836 provider sites, 19,562 (94 %) were enrolled in the VFC program. 

The 19,562 VFC-Enrolled provider sites that were visited constitute 46% of the total 42,073 Active VFC-Enrolled provider sites. Table 3 illustrates the percentage of visits for each site type (Public, C/MHC, Other Public and Private). 

Table 2. Number of Provider Sites Receiving at Least 1 visit, CY2002
Provider Information	Public	C/MHC*	Other Public	Subtotal All Public	Private	Total	
VFC-Enrolled Provider Sites 	3344	1864	953	6161	13,401	19,562	
Non-VFC Provider Sites 					1274	1274	
Total 	3344	1864	953	6161	14,675	20,836	
*Community or Migrant Health Center

Table 3. Number and Percent of VFC-enrolled Provider Sites Receiving at Least 1 Visit, CY2002
Provider Information	Public	C/MHC	Other Public	Subtotal All Public	Private	Total	
Number of VFC-Enrolled Provider Sites	5094	3802	2755	11,651	30,422	42,073	
Number (Percent) of VFC-Enrolled Providers who Received at Least 1 Visit During CY2002	3344 (66%)	1862 (49%)	953    (35%)	6161
(53%)	13,401 (44%)	19,562
(46%)	

Site Visit Information
While 19,562 VFC-Enrolled provider sites were visited in 2002, the actual number of VFC Only, AFIX Only, VFC/AFIX Combined, and Educational visits to a VFC-Enrolled Provider site totaled 23,290. Table 4 below details the number of visits to VFC and Non-VFC provider sites by visit type and provider type. Essentially 8131 visits were conducted in public VFC Enrolled sites and 15,159 visits were conducted in private VFC Enrolled sites. In addition, 205 visits (AFIX Only and Educational) were conducted in non-VFC Enrolled provider sites. 

Table 4.  Total number of visits by provider type, CY2002
Type of Visit	Public
VFC Enrolled	Public Non-VFC	C/MHC VFC Enrolled	C/MHC Non-VFC	Other Public VFC Enrolled	Other Public Non-VFC	Private VFC Enrolled	Private Non-VFC	
VFC Only*	1039		448		243		4431		
AFIX Only**	960	64	174	2	49	0	1330	120	
VFC/AFIX Combined***	2551		1377		234		7129		
Educational	408	2	469	0	179	0	2269	17	
* VFC Only is defined as a visit to a VFC enrolled provider to ensure compliance with VFC program requirements. 
** AFIX Only is defined as a quality improvement strategy utilizing assessment of immunization records, feedback, incentive, and exchange of information through performance measurement, diagnosis of service delivery problems, and data feedback during a visit to a medical practice.  One AFIX visit should contain an assessment and a feedback component even though more than one physical visit to the provider site may be required to complete the assessment and the feedback session. 
*** A VFC/AFIX Combined site visit is defined as a visit to a VFC-Enrolled provider site which integrates the review to ensure compliance with VFC program requirements and immunization record assessment and feedback activities. 

Table 5 includes additional documentation regarding site visits in CY 2002. Repeat AFIX visits are a subset of the AFIX visits documented in Table 4 and they are included to illustrate the number of visits that occurred at a site that had previously received an AFIX visit between January 1, 2000 and December 31, 2001. The nature of AFIX as a continuous quality improvement strategy requires that provider sites are visited on more than one occasion to evaluate incremental progress. The number of repeat AFIX visits allows CDC/NIP to track the grantees progress with implementing this ongoing quality improvement strategy. The VFC Follow-Up visits describe the number of visits that occurred as a result of a problem and/or concern found at the initial VFC visit. The information in Tables 4 and 5 reveal that grantees are actively visiting provider sites, following up on problems identified in previous visits, providing education as well as service, and ultimately building relationships with staff.  

Table 5. Additional Visits, CY2002
Type of Visit	Public
VFC Enrolled	Public Non-VFC	C/MHC VFC Enrolled	C/MHC Non-VFC	Other Public VFC Enrolled	Other Public Non-VFC	Private VFC Enrolled	Private Non-VFC	
Repeat AFIX*	1263	1	376	0	82	0	1465	80	
Follow-up VFC**	545		49		8		560		
*Repeat AFIX: the number of AFIX visits (includes AFIX only and Combined VFC/AFIX) from Table 4 that are repeat assessments (e.g. the provider has received an assessment during a previous year).
**Follow-up VFC: the number of visits completed to evaluate provider response to previously identified problems found during the initial VFC site visit.  

As part of the annual grant application process, grantees are required to specify the proposed number of site visits to be conducted in the upcoming calendar year. For the 2002 Grant Applications, the planned number of site visits included three categories: VFC Only, AFIX Only and VFC/AFIX Combined. In an effort to examine the accuracy with which a grantee can estimate VFC/AFIX activities, the number of proposed site visits from the grant applications were compared with the actual number of site visits (for the categories of VFC Only, AFIX Only and VFC/AFIX Combined). Grantees surpassed the total number of proposed site visits in CY 2002 by 914 visits (see Table 6).   

Table 6.  Proposed and Actual Number of Site Visits, CY2002. 
Type of Site Visit	Public Provider*	Private Provider	Total Visits	
	Proposed	Actual	Proposed	Actual	Proposed	Actual	
VFC only	745	1730	3362	4431	4107	6161	
AFIX only	916	1183	679	1330	1595	2513	
VFC/AFIX combined	4699	4162	8650	7129	13,349	11,291	
Total visits	6360	7075	12,691	12,890	19,051	19,965	
*Public provider includes Public, C/MHC, and Other Public.

Assessment Outcome Measures for Public Providers
Of the 61 grantees that receive funds for VFC-AFIX activities, 58 submitted public provider data for this annual report (see Appendix A). Of those 58 that submitted data, 46 use the CASA, Mini-CASA or equivalent method (designated as “CASA”). Two grantees use the Hybrid method, and 10 grantees use a combination of CASA and Hybrid (designated as “Both). The difference between the CASA and Hybrid methods is found in the results from the assessment. Grantees that use CASA as the assessment method will receive an estimate coverage level for that provider. Grantees that use the Hybrid assessment method will receive a result indicating the provider is “above” or “below” a pre-determined threshold level of coverage. (See Appendix B for more in-depth explanation.) 

The majority of the grantees (25 of 58) assess children 24-35 months of age. Other age groups reported include 19-35 months (11 grantees), 12-23 months (2 grantees), and 6 grantees reported that they assess both ages 12-23 and 24-35 months. Thirteen grantees responded to this question with “other” and one grantee did not report this information.

For those grantees using CASA for the assessment, six grantees reported provider vaccination coverage levels averaging 80% or higher. Twenty-one grantees reported provider vaccination coverage levels averaging 70-79%.  Of the remaining results, 12 grantees indicated vaccination coverage levels averaging 60-69% and 16 indicated vaccination coverage levels below 60%. One grantee did not report this information. 

For those providers using the Hybrid Method, threshold levels of 60, 70, 75, 80 and 90 percent were used. The most common threshold levels were 70 and 80 percent.  For those grantees that used the Hybrid methodology, only 2 grantees reported the number of providers “passing”, or at or above the threshold, to be greater than 80% of the providers assessed. 

Note: The above numbers are not mutually exclusive – meaning one grantee could have done some AFIX visits using the Hybrid method and some visits using the CASA, Mini-CASA, or Equivalent method. 

Assessment Outcome Measures for Private Providers
Of the 61 grantees that receive funds for VFC-AFIX activities, 57 submitted private provider data for this annual report (see Appendix C). Of those 57 that submitted data, 43 use the CASA, Mini-CASA or equivalent method (designated as “CASA”). Two grantees use the Hybrid method, and 11 grantees use a combination of CASA and Hybrid (designated as “Both”). 

The majority of the grantees (25 of 57) assess children 24-35 months of age. Other age groups reported include 19-35 months (11 grantees), 12-23 months (2 grantee), and 6 grantees reported that they assess children ages 12-23 and 24-35 months of age. Twelve grantees responded to this question with “other” and 1 grantee did not report this information.

For those grantees using CASA for the assessment, 8 grantees reported provider vaccination coverage levels averaging 80% or higher. Seventeen grantees reported provider vaccination coverage levels averaging 70-79%.  For the remainder, 17 grantees indicated vaccination coverage levels averaging 60-69% and 12 indicated vaccination coverage levels below 60%.  One grantee did not report this information. 

For those providers using the Hybrid Method, threshold levels of 60, 70, 75, 80 and 90 percent were used. The most common threshold levels were 75 and 80 percent.  One grantee reported the number of providers “passing”, or at or above the threshold, represented greater than 80% of the providers assessed using the Hybrid method.  

Note: The above numbers are not mutually exclusive – meaning one grantee could have done some AFIX visits using the Hybrid method and some visits using the CASA, Mini-CASA, or equivalent method. 

Change in Coverage when using CASA to assess Public Providers
Thirty-one grantees used the CASA method to assess coverage during previous and CY2002 assessments in the public sector (see Appendix D). The majority (18 of 29) assessed for 4 Diphtheria, Tetanus, and Pertussis (DTaP) doses; 3 Polio doses (IPV or OPV); 1 Measles, Mumps and Rubella dose (MMR), 3 Haemophilus Influenzae Type B doses (Hib) and 3 Hepatitis B doses (HepB) for previous and CY 2002 assessments. This series of vaccinations is documented as 4:3:1:3:3. Twelve grantees assessed for 4 doses of DTaP, 3 doses of Polio and 1 dose of MMR for previous and CY 2002 assessments; commonly referred to as the 4:3:1 series. One grantee assessed for the 4:3:1:3 series for previous and CY 2002 assessments, which is 4 doses of DTaP, 3 doses of Polio, 1 dose of MMR, and 3 doses of Hib. One grantee assessed for the 3:2:2:2 series for previous and CY 2002 assessments, which is 3 doses of DTaP, 2 doses of Polio, 2 doses of Hib and 2 doses of HepB. 

Four grantees documented higher average public provider vaccination coverage at 10% or greater. Eight grantees found the average public provider vaccination levels improved by 1-9%.  The average public provider vaccination coverage levels did not change for 3grantees and the change was negative for 16 grantees. This negative change ranged from -1% to -10% for all but three grantees. The other three grantees observed the average percentage point change in coverage to be -10.1%, -18% and -23%.

Grantees that demonstrated either a significant increase or decrease in change in coverage were contacted to elicit possible factors for this change in coverage.  A decrease in coverage was most often attributed to changes in the data collection procedures and policies. For one grantee, the sampling technique was changed and as a result, the rates decreased; however, this grantee indicated that the results are more accurate as a result of the change and will be comparable with future assessments.  Another reason provided for the decrease in rates was the deferral of the 4th DTaP during the vaccine shortage. Assessing compliance with all recommended vaccines by 24 months of age became more difficult for this reason. In addition, the population of the children served in the public sector is very small in some geographic areas. As a result, small fluctuations in the number of children that are in compliance with their immunizations may actually cause what appears to be a large difference in coverage levels. 

Change in Coverage when using CASA to assess Private Providers
Thirty grantees used the CASA method to assess private provider coverage levels during previous and CY2002 assessments in the private sector (see Appendix E). The majority (16 of 29) assessed for the 4:3:1:3:3 series for previous and CY 2002 assessments. Twelve grantees assessed for the 4:3:1 series and 2 grantees assessed for the 4:3:1:3 series. 

Nine grantees documented change in the average private provider vaccination level to be 10% or greater. Eleven grantees found the average public provider vaccination level to change between 1-9%.  The average private provider vaccination coverage levels did not change for 2 grantees and the change was negative for 7 grantees. This negative change ranged from -.8% to -3.7%.

Grantees that demonstrated either a significant increase or decrease in change in coverage were contacted to elicit possible factors for this change in coverage. The responses from the grantees regarding decreases in coverage included more stringent data collection methods in CY 2002 than in previous years, improvements in data management in CY 2002, and selected vaccine shortages impacting the timely completion of the required series. The responses from the grantees regarding increases in coverage described activities that focused on building relationships with the providers, marketing VFC and AFIX visits as “a service” not an “audit” and establishing public health staff as a “resource” rather than a “regulator”. 

Hybrid Method Results
Only one grantee used the Hybrid method in the public sector for previous and CY2002 assessments (see Appendices F). Two grantees used the Hybrid method in the private sector for previous and CY2002 assessments (see Appendices G). The grantees assessed for the vaccination series 43133 with a threshold level of 80. 

VFC Accountability Results
At least 25% of VFC private providers should be visited annually; 83% (49 of 59) of the grantees that submitted data for the “Annual VFC Management Survey” met this goal for CY2002. This report documents the first time that data regarding the percent of VFC enrolled provider sites receiving a VFC Visit were collected. Therefore, CY2002 marks the baseline measurement of the percentage of providers receiving a VFC visit. This area of quality assurance will need to be analyzed in detail over time to determine the reasons behind these results. 

2002 NIP Training and Education Activities
NIP staff working on the VFC/AFIX project focused in 2002 on improving grantee training and education.  The following activities were implemented by NIP in 2002 to enhance training:
•	VFC/AFIX staff created a training request application and posted it on the AFIX website.  All grantees or organizations are required to complete the application before training will be scheduled.  When an application is received the VFC/AFIX staff schedules a conference call with the grantee to discuss content and logistics of the training.  After the call is completed, the grantee receives a draft agenda of the proposed training and draft presentations from NIP staff to ensure that training content meets the grantee’s identified needs.
•	Customizing trainings for grantees have allowed the VFC-AFIX staff to develop a large presentation library that can be made available to grantees so they can utilize the presentations for local training.
•	Evaluations are systematically conducted at all NIP conducted trainings.  The results of the participant evaluations are shared among the participating NIP staff and the grantee.

NIP staff conducted AFIX and/or CASA trainings on site to the following grantees in 2002:
•	Louisiana
•	North Carolina
•	South Carolina
•	New York State

The following grantees requested educational presentations or limited trainings on specific aspects of AFIX or CASA in 2002:
•	Maryland
•	Texas
•	Florida

Summary of Findings, Current Challenges and Future Directions
The second year of full implementation for the VFC/AFIX Project was a transition year for the program.  A vision for the VFC/AFIX initiative was established at the federal level. This initiative has progressed from a theoretical concept to a dynamic, evolving public health program.  At the state and local level, grantees began to measure changes in coverage levels both in the public and private sector. Many grantees also focused on improving the implementation of the Incentives and eXchange of information components of the AFIX strategy. 

The 2002 data reported by the grantees to NIP reflects coverage level improvements from 2001 to 2002 in private provider offices for the majority of the grantees. The data also suggest that selected grantees may need further or more detailed assistance/instruction on the use of the VFC/AFIX evaluation software.  Additionally, selected grantees may need more individualized technical assistance on how to move this project from a conceptual basis to an active dynamic public health program. For specifics on activities planned for CY2003, please refer to the “Proposed VFC-AFIX Activities for CY 2003” report submitted to OMB on February 1, 2003. 

Many new external challenges faced both grantee and federal staff working on the VFC-AFIX program that were unknown when this program was expanded to all eligible grantees in 2001. Shortages of many childhood vaccines occurred in 2002, adding a confounding variable to improving immunization coverage levels. This shortage however, also provided many grantees the opportunity to discuss proven immunization quality improvement strategies with providers, including methods to recall children who had immunizations deferred.  From a global perspective, the threat of bio-terrorism and the need to develop preparedness strategies, plans, and programs caused a realignment of human resource priorities for significant periods in 2002 and early 2003. On a human resource level, state imposed budgetary restrictions and restraints affected some grantees’ ability to efficiently hire or replace staff and the ability to travel and/or bring staff together for training purposes. Despite the challenges that occurred in 2002 and that continue to occur in 2003, the VFC/AFIX project continues to evolve and has begun to show successful outcomes in the private sector.

APPENDIX A 

Appendix A: Assessment Outcome Measures for Public Providers by Grantee, CY2002
Grantee	Number Assessed with CASA1	Age Group Assessed (months)2	
Vaccination Series Measured3	Assessment Method4	Minimum Coverage Level5	Maximum Coverage Level6	Crude Average Coverage Level7	Vaccination Series Assessed with Hybrid8 	
Number Assessed with Hybrid9	
Hybrid Threshold Level10	Number of Providers At or Above Threshold Level11	
Alabama	5	other	4:3:1	both	50	94	75.2	4:3:1	6	80	2	
Alaska	26	12 & 24	4:3:1:3:3	casa	26%	88	59					
Arizona	6	19-35	4:3:1:3:3	both	38	83	58.2	4:3:1:3:3	3	75	1	
Arkansas	95	24-35	4:3:1:3:3	casa			76.77					
California	378	other	4:3:1:3:3	both	10.4	100	64.4	various	1	80	various	
Chicago	41	12&24	4:3:1:3:3	casa	13.3	80	47.8					
Colorado	35	19-35	4:3:1:3:3	both	17.5	96	53.5	4:3:1:3:3	56	75	16	
Connecticut	34	19-35	4:3:1:3:3	casa	44	100	71					
Delaware	2	12&24	4:3:1:3:3	casa	33.3	34	33.65					
District Of Columbia	6	12&24	4:3:1:3:3	casa	44	67	59					
Florida	163	24-35	4:3:1:3:3	casa	18	100	75.9					
Georgia	253	24-35	4:3:1	casa	0	100	91					
Hawaii	12	24-35	4:3:1:3:3	both	0	100	62	4:3:1:3:3	2	80	1	
Houston	11	other	4:3:1:3:3	casa	39	92	65					
Idaho	17	19-35	4:3:1	casa	44	95	82.5					
Illinois	133	12&24	4:3:1:3	casa	21	100	60.5					
Indiana	195	19-35	4:3:1:3	casa	0	100	79					
Iowa	134	other	4:3:1:3:3	casa	24	100	84					
Kansas	124	other	4:3:1:3:3	both	20	100	63	4:3:1:3:3	6	60	5	
Kentucky	163	24-35	4:3:1:3:3	casa	31	100	84.1					
Louisiana	109	24-35	4:3:1	both	20	100	76.8	4:3:1	34	90	15	
Maine	51	24-35	4:3:1:3:3	casa	36	100	76.1					
Maryland	42	24-35	4:3:1:3:3	casa	41.7	100	76					
Massachusetts		24-35		hybrid				4:3:1:3:3	51	80	20	
Grantee	Number Assessed with CASA1	Age Group Assessed2	
Vaccination Series Measured3	Assessment Method4	Minimum Coverage Level5	Maximum Coverage Level6	Crude Average Coverage Level7	Vaccination Series Assessed with Hybrid8 	
Number Assessed with Hybrid9	
Hybrid Threshold Level10	Number of Providers At or Above Threshold Level11	
Michigan	89	19-35	4:3:1:3:3	casa	0	0.01	0					
Minnesota	11	24-35	4:3:1:3:3	casa	0	80.6	33.7					
Mississippi	215	other	4:3:1	casa	0	100	74.9					
Missouri	175	24-35	4:3:1	casa	20	100	71.3					
Montana		24-35		hybrid				4:3:1:3:3				
Nebraska	74	19-35	4:3:1:3:3	both	16	100	70	4:3:1:3:3	2	70	2	
Nevada	18	24-35	4:3:1:3:3	casa	11	92	63					
New Hampshire	45	24-35	4:3:1:3:3	casa	0	100	79					
New Jersey	91	other	4:3:1:3:3	casa	0	100	75					
New Mexico	58	24-35	4:3:1:3:3	casa	24	88	61					
New York City	20	24-35	4:3:1:3:3	casa	45	89	71					
New York State	67	24-35	4:3:1:3:3	casa	20	100	70	4:3:1:3:3	6	70	3	
North Carolina	100	12&24	4:3:1	casa	24	100	79					
North Dakota	17	24-35	4:3:1:3:3	casa	39	100	63					
North Mariana Islands	3		4:3:1:3:3	casa								
Ohio	69	24-35	4:3:1:3:3	casa	21	89	61					
Oklahoma	53	24-35	4:3:1:3:3	casa	20	83	55					
Oregon	48	other	4:3:1:3:3	casa		75	56.2					
Pennsylvania	94	24-35	4:3:1:3:3	casa	64.7	80.5	71.7					
Philadelphia	12	24-35	4:3:1:3:3	casa	30	96	59.1					
Puerto Rico	220	other	3:2:2:2	casa	4	100	52					
Rhode Island	15	12-23	3:2:2:2	casa	74	100	84.5					
San Antonio	21	other	4:3:1:3:3	casa	14.3	93	63.3					
South Carolina	16	24-35	4:3:1:3:3	casa	37.4	88.1	65.7					
South Dakota	65	19-35	4:3:1	casa	50	100	90					
Tennessee	105	other	4:3:1:3:3	both	18	77	45.1	4:3:1:3:3	3	70	0	
Grantee	Number Assessed with CASA1	Age Group Assessed2	
Vaccination Series Measured3	Assessment Method4	Minimum Coverage Level5	Maximum Coverage Level6	Crude Average Coverage Level7	Vaccination Series Assessed with Hybrid8 	
Number Assessed with Hybrid9	
Hybrid Threshold Level10	Number of Providers At or Above Threshold Level11	
Texas	585	other	4:3:1	casa	4	100	64.31					
Utah	46	19-35	4:3:1:3:3	casa	0	100	78.8					
Vermont	3	24-35	4:3:1	casa	33	97	72.3					
Virginia	35	12-23	4:3:1:3:3	casa	8	80	54					
Washington	23	19-35	4:3:1:3:3	both	17	100	52.2	4:3:1:3:3	15	70	3	
West Virginia	31	24-35	4:3:1	casa	0	100	72.2					
Wisconsin	11	19-35	4:3:1:3:3	casa	2	100	55.3					
Wyoming	42	other	4:3:1	casa	75	100	79					

1 Number Assessed with CASA: number of providers that received an assessment using the CASA method.
2 Age Group Assessed: the age range of the patients included in the assessment
3 Vaccination Series Measured: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 
4 Assessment Method: method used to evaluate vaccine coverage level (CASA or Hybrid) 
5 Minimum Coverage Level: among all providers assessed, the lowest determined vaccination coverage for the vaccination series measured
6 Maximum Coverage Level: among all providers assessed, the highest determined vaccination coverage for the vaccination series measured
7 Crude Average Coverage Level: an unweighted average vaccination coverage level among all provider sites
8 Vaccination Series Assessed with Hybrid: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit when using the Hybrid Method.  
9 Number Assessed with Hybrid: number of providers that received an assessment using the CASA method.
10 Hybrid Threshold Level: vaccination coverage level at which providers are expected to be performing at or above
11 Number of Providers At or Above Threshold Level: number of providers at or above threshold level 

APPENDIX B 

Appendix B: Assessment Methodology Options

Method	
Description	
Advantages	
Disadvantages	

Diagnostic Tools	
CASA	
Sample selection: random sample
Sample size: approximately 100 per cohort
Inputs:	child’s demographic information; date of each immunization; other information related to diagnostic analysis
Outputs:	diagnostic information on late starts, drop-offs and missed opportunities
baseline measure of coverage	
Precise estimates of immunization coverage levels; evaluation of missed opportunities; evaluation of late starts, etc.	
Large sample size
Resource burden (staff, time).	
	
Registry based	
Sample selection: census of all  eligible records
Sample size: all eligible records
Inputs:	data from registry downloaded to CASA 
Outputs:	diagnostic information on late starts, drop-outs and missed opportunities	
Minimal time and effort for data collection.
No sampling error since estimates based on census of records. 	
Potential biases:
- Registry may not contain 100% of provider’s records.
- Reliability of registry data	
	
Mini-CASA	
Sample selection:  consecutive, convenience or random sample
Sample size: 40-60
Inputs: 	same as CASA
Outputs:	diagnostic information on late starts, drop-off and missed opportunities	
Smaller sample sizes.	
Coverage estimates have less precision; sample may not be randomly selected.	

Hybrid	
Hybrid
(LQA/CASA) 	
Sample selection: random sample
Sample size: 30
Inputs:	same as CASA
Outputs:	determines if a provider has immunization coverage above or below a specified threshold
                 vaccine histories of not up-to-date clients as examples to                     discuss with provider and staff	
Smallest sample size
Rapid assessment
More feedback information than LQA alone.
Identifies providers who may benefit from a diagnostic assessment.	
Computer needed (not paper and pencil as LQA).
Does not give point estimate of coverage.
Smaller basis for diagnostic feedback.	

APPENDIX C

Appendix C: Assessment Outcome Measures for Private Providers by Grantee, CY2002

Grantee	Number Assessed
with CASA1	Age Group Assessed2	
Vaccination Series Measured3	Assessment Method4	Minimum Coverage Level5	Maximum Coverage Level6	Crude Average Coverage Level7	Vaccination Series Assessed with Hybrid8	
Number Assessed with Hybrid9	
Hybrid Threshold Level10	Number of Providers At or Above Threshold Level 11	
Alabama	195	other	4:3:1	both	0	100	82.8	4:3:1	1	80	0	
Alaska	7	12&24	4:3:1:3:3	casa	45	73	59					
Arizona	109	19-35	4:3:1:3:3	both	0	100	59.8	4:3:1:3:3	37	70	16	
Arkansas	73	24-35	4:3:1:3:3	casa			28.35					
California	112	other	4:3:1:3:3	both	10	100	65.5		144	80		
Chicago	99	12&24	4:3:1:3:3	both	0	96	56.2	4:3:1	37	75	18	
Colorado	22	19-35	4:3:1:3:3	both	23.5	100	63.6	4:3:1:3:3	49	75	14	
Connecticut	101	19-35	4:3:1:3:3	casa			84					
Delaware	25	12&24	4:3:1:3:3	casa	59.7	96.8	80.3					
District of Columbia	23	12&24	4:3:1:3:3	casa	30	94	62					
Florida	276	24-35	4:3:1:3:3	casa	31	100	77.7					
Georgia	158	24-35	4:3:1	casa	74	100	94					
Hawaii	44	24-35	4:3:1:3:3	both	0	100	75	4:3:1:3:3	9	80	3	
Houston	59	other	4:3:1:3:3	both	0	94	65	4:3:1:3:3	59	70	12	
Idaho	34	19-35	4:3:1	casa	0	100	71.5					
Illinois	361	12&24	4:3:1:3	casa	27	100	63.5					
Indiana	508	19-35	4:3:1:3	casa	9	100	79.2					
Iowa	57	other	4:3:1:3:3	casa	43	100	79					
Kansas	15	other	4:3:1:3:3	both	0	100	50	4:3:1:3:3	3	60	2	
Kentucky	64	24-35	4:3:1:3:3	casa	50	100	78.8					
Louisiana	112	24-35	4:3:1	both	10	100	67.1	4:3:1	51	90	21	
Maine	110	24-35	4:3:1:3:3	casa	0	100	72.5					
Maryland	534	24-35	4:3:1:3:3	casa	0	100	87					
Grantee	Number Assessed
with CASA1	Age Group Assessed2	
Vaccination Series Measured3	Assessment Method4	Minimum Coverage Level5	Maximum Coverage Level6	Crude Average Coverage Level7	Vaccination Series Assessed with Hybrid8	
Number Assessed with Hybrid9	
Hybrid Threshold Level10	Number of Providers At or Above Threshold Level 11	
Massachusetts		24-35		hybrid				4:3:133	359	80	208	
Michigan	111	19-35	4:3:1:3:3	casa	11	100	67					
Minnesota	102	24-35	4:3:1:3:3	casa	4.6	100	64.4					
Mississippi 	63	other	4:3:1	casa	0	100	73.5					
Missouri	120	24-35	4:3:1	casa	22	100	75.3					
Montana		24-35		hybrid				4:3:1:3:3				
Nebraska	122	19-35	4:3:1:3:3	both	0	100	66	4:3:1:3:3	41	70	38	
Nevada	21	24-35	4:3:1:3:3	casa	1	82	46					
New Hampshire	69	24-35	4:3:1:3:3	casa	0	100	79					
New Jersey	127	other	4:3:1:3:3	casa	0	100	62.5					
New Mexico	76	24-35	4:3:1:3:3	casa	18	89	61					
New York City	22	24-35	4:3:1:3:3	casa	9	93	58					
New York State	293	24-35	4:3:1:3:3	casa	0	100	74					
North Carolina	104	12&24	4:3:1	casa	31	100	74.1					
North Dakota	8	24-35	4:3:1:3:3	casa	37	82	63					
North Mariana Islands	4		4:3:1:3:3	casa								
Ohio	60	24-35	4:3:1:3:3	casa			68.9					
Oklahoma	15	24-35	4:3:1:3:3	casa	33	92	71					
Oregon	79	other	4:3:1:3:3	casa	12.7	80.1	50.1					
Pennsylvania	521	24-35	4:3:1:3:3	casa	76.4	85.6	81.6					
Philadelphia	113	24-35	4:3:1:3:3	casa	0	100	55.3					
Rhode Island	42	12-23	3:2:2:2	casa	0	100	88.5					
San Antonio	81	other	4:3:1:3:3	casa	0	100	59.7					
South Carolina	10	24-35	4:3:1:3:3	casa	24.8	93.9	62					
South Dakota	119	19-35	4:3:1	casa	0	100	74					
Tennessee	76	other	4:3:1:3:3	both	4	100	69.7	4:3:1:3:3	23	90	14	
Grantee	Number Assessed with CASA1	Age Group Assessed2	
Vaccination Series Measured3	Assessment Method4	Minimum Coverage Level5	Maximum Coverage Level6	Crude Average Coverage Level7	Vaccination Series Assessed with Hybrid8 	
Number Assessed with Hybrid9	
Hybrid Threshold Level10	Number of Providers At or Above Threshold Level11	
Texas	1130	other	4:3:1	casa	4	100	74					
Utah	106	19-35	4:3:1:3:3	casa	0	96.9	61.5					
Vermont	25	24-35	4:3:1	casa	71	100	88.5					
Virginia	66	12-23	4:3:1:3:3	casa	0	96	67					
Washington	75	19-35	4:3:1:3:3	both	11	92	52.7	4:3:1:3:3	51	70	29	
West Virginia	33	24-35	4:3:1	casa	35	100	79.2					
Wisconsin	55	19-35	4:3:1:3:3	casa	0	97	59.4					
Wyoming	50	other	4:3:1	casa	50	100	76					

1 Number Assessed with CASA: number of providers that received an assessment using the CASA method.
2 Age Group Assessed: the age range of the patients included in the assessment
3 Vaccination Series Measured: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 
4 Assessment Method: method used to evaluate vaccine coverage level (CASA or Hybrid) 
5 Minimum Coverage Level: among all providers assessed, the lowest determined vaccination coverage for the vaccination series measured
6 Maximum Coverage Level: among all providers assessed, the highest determined vaccination coverage for the vaccination series measured
7 Crude Average Coverage Level: an unweighted average vaccination coverage level among all provider sites
8 Vaccination Series Assessed with Hybrid: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit when using the Hybrid Method.  
9 Number Assessed with Hybrid: number of providers that received an assessment using the CASA method.
10 Hybrid Threshold Level: vaccination coverage level at which providers are expected to be performing at or above
11 Number of Providers At or Above Threshold Level: number of providers at or above threshold level 

APPENDIX D 

Appendix D

Change in Vaccination Coverage Levels, Grantees That Used CASA (or Equivalent) Method for Previous and CY2002 Assessments for Public Providers (n=31)

Grantee	Total Number Assessed1	
Vaccination Series Measured2
	
Crude Average Coverage Level (Previous)3
	Crude Average Coverage Level (CY2002)4	Average Percentage Point Change in Coverage5	
Alaska	43	4:3:1:3:3	66.8	60.6	-6.2	
Arkansas	98	4:3:1:3:3	81	76.77	-4.2	
Chicago	41	4:3:1	42.8	51.5	8.7	
Connecticut	217	4:3:1:3:3	72	71	-1	
Delaware	36	4:3:1:3:3	35.3	33.6	-1.7	
Florida	53	4:3:1:3:3	76	75.9	-.1	
Georgia	256	4:3:1	94	91	-3	
Hawaii	10	4:3:1:3:3	70	62	-8	
Houston	80	4:3:1:3:3	65	65	0	
Idaho	4	4:3:1	57.3	88.3	31	
Indiana	664	4:3:1:3	82	79	-3	
Iowa	180	4:3:1:3:3	88	84	-4	
Kansas	236	4:3:1:3:3	77	59	-18	
Louisiana	71	4:3:1	85.3	82.5	-2.8	
Maine	80	4:3:1:3:3	69.3	75.6	6.3	
Michigan	132	4:3:1:3:3	23	42	19	
Mississippi	128	4:3:1	72.8	73.4	.6	
Missouri	440	4:3:1	78.2	73.1	-5.1	
Nevada	50	4:3:1:3:3	59	63	4	
New Mexico	189	4:3:1:3:3	62	61	-1	
New York City	12	4:3:1:3:3	47	78	31	
New York State	40	4:3:1:3:3	66.5	73	6.5	
North Carolina	111	4:3:1	78	79	1	
Puerto Rico	214	3:2:2:2	75	52	-23	
San Antonio	91	4:3:1:3:3	58	60	2	
South Dakota	184	4:3:1	91	90	-1	
Texas		4:3:1	64	64	0	
Utah	53	4:3:1:3:3	88.9	78.8	-10.1	
Vermont	15	4:3:1	0	0	0	
West Virginia	48	4:3:1	47	75.2	28.2	
Wyoming		4:3:1	78	79	1	

1Total Number Assessed: total number of providers that received an AFIX visit
2 Vaccination Series Measured: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 
3 Crude Average Coverage Level (Previous): an unweighted average vaccination coverage level among all provider sites based on vaccination coverage levels from assessments performed between 1/01/000 and 12/31/01
4 Average Coverage Level (CY 2002): an unweighted average vaccination coverage level among all provider sites based on vaccination coverage levels from assessments performed during CY 2002
5 Average Percentage Point Change in Coverage: increase or decrease between previous coverage level and coverage level CY 2002

APPENDIX E

Appendix E

Change in Vaccination Coverage Levels, Grantees That Used CASA (or Equivalent) Method for Previous and CY2002 Assessments for Private Providers (n=30)

Grantee	
Total Number Assessed1	
Vaccination Series Measured2
	
Crude Average Coverage Level (Previous)3
	
Crude Average Coverage Level (CY2002)4
	Average Percentage Point Change in Coverage5	
Alaska	43	4:3:1:3:3	56	59.1	3.1	
Chicago	41	4:3:1	46.4	55.1	8.7	
Connecticut	217	4:3:1:3:3	87	84	-3	
Delaware	36	4:3:1:3:3	62.7	80.3	17.6	
Florida	53	4:3:1:3:3	79	77.7	-1.3	
Georgia	256	4:3:1	76	94	18	
Houston	80	4:3:1:3:3	26	44	18	
Idaho	4	4:3:1	77.5	75	-2.5	
Indiana	664	4:3:1:3	73.5	79.2	5.7	
Iowa	180	4:3:1:3:3	74	79	5	
Kansas	236	4:3:1:3:3	66	66	0	
Louisiana	71	4:3:1	0	0	0	
Maine	80	4:3:1:3:3	72.7	69	-3.7	
Maryland	136	4:3:1:3:3	79.7	85.2	5.5	
Michigan	132	4:3:1:3:3	60	73	13	
Mississippi	128	4:3:1	68.9	77.5	8.6	
Missouri	440	4:3:1	77.5	74.7	-2.8	
Nevada	50	4:3:1:3:3	36	46	10	
New Mexico	189	4:3:1:3:3	57	61	4	
New York City	12	4:3:1:3:3	49	67	18	
New York State	40	4:3:1:3:3	67.4	69.7	2.3	
North Carolina	111	4:3:1	68	69	1	
San Antonio	91	4:3:1:3:3	44	59	15	
South Dakota	184	4:3:1	72	74	2	
Tennessee	10	4:3:1:3	73.6	93.8	20.2	
Texas		4:3:1	57	59	2	
Utah	53	4:3:1:3:3	54.7	61.4	6.7	
Vermont	15	4:3:1	92.3	89.1	-3.2	
West Virginia	48	4:3:1	76	75.2	-.8	
Wyoming		4:3:1	65	76	11	

1Total Number Assessed: total number of providers that received an AFIX visit
2 Vaccination Series Measured: vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. All coverage levels will refer to the completion of this series. 
3 Crude Average Coverage Level (Previous): an unweighted average vaccination coverage level among all provider sites based on vaccination coverage levels from assessments performed between 1/01/000 and 12/31/01
4 Average Coverage Level (CY 2002): an unweighted average vaccination coverage level among all provider sites based on vaccination coverage levels from assessments performed during CY 2002
5 Average Percentage Point Change in Coverage: increase or decrease between previous coverage level and coverage level CY 2002

APPENDIX F 

Appendix F

Improvement in the Number of Providers with Vaccination Coverage Above the Threshold Level, Grantees That Used Hybrid Method for Previous and CY2002 Assessments for Public Providers (n=1)

Grantee	Total Number Assessed (Previous)1	Vaccination Series Measured (Previous) 2	Threshold Level (Previous)3	
Number (%) of Providers At or Above Threshold Level (Previous)4	Total Number Assessed (CY 2002) 5	Vaccination Series Measured (CY 2002) 6	Threshold Level (CY 2002) 7	
Number (%) of Providers At or Above Threshold Level (CY 2002) 8
	
Massachusetts	42	4:3:1:3:3	80	26	42	4:3:1:3:3	80	16	

1Total Number Assessed (Previous): total number of providers that received an assessment with the Hybrid method prior to 2002
2 Vaccination Series Measured (Previous): vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. 
3 Threshold Level (Previous): vaccination coverage level at which providers are expected to be performing at or above for assessments performed for previous assessments
4 Number (%) of Providers At or Above Threshold Level (Previous): number of providers at or above threshold level based on vaccination coverage levels from assessments performed between 1/1/00 and 12/31/01
5 Total Number Assessed (CY2002): total number of providers that received an assessment with the Hybrid method in CY 2002
6 Vaccination Series Measured (CY 2002): vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. 
7 Hybrid Threshold Level (CY2002): vaccination coverage level at which providers are expected to be performing at or above for assessments performed during CY 2002
8 Number (%) of Providers At or Above Threshold Level (CY 2002): number of providers at or above threshold level based on vaccination coverage levels from assessments performed during CY 2002

APPENDIX G 
Appendix G

Improvement in the Number of Providers with Vaccination Coverage Above the Threshold Level, Grantees That Used Hybrid Method for Previous and CY2002 Assessments for Private Providers (n=2)

Grantee	Total Number Assessed (Previous)1	Vaccination Series Measured (Previous) 2	Threshold Level (Previous)3	
Number (%) of Providers At or Above Threshold Level (Previous)4	Total Number Assessed (CY 2002) 5	Vaccination Series Measured (CY 2002) 6	Threshold Level (CY 2002) 7	
Number (%) of Providers At or Above Threshold Level (CY 2002) 8
	
California	47	Various	80	24	42	Various	80	21	
Massachusetts	253	4:3:1:3:3	80	136	253	4:3:1:3:3	80	146	

1Total Number Assessed (Previous): total number of providers that received an assessment with the Hybrid method prior to 2002
2 Vaccination Series Measured (Previous): vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. 
3 Threshold Level (Previous): vaccination coverage level at which providers are expected to be performing at or above for assessments performed for previous assessments
4 Number (%) of Providers At or Above Threshold Level (Previous): number of providers at or above threshold level based on vaccination coverage levels from assessments performed between 1/1/00 and 12/31/01
5 Total Number Assessed (CY2002): total number of providers that received an assessment with the Hybrid method in CY 2002
6 Vaccination Series Measured (CY 2002): vaccine series (type and number of doses of vaccine) used to evaluate up-to-date status during the assessment visit. 
7 Hybrid Threshold Level (CY2002): vaccination coverage level at which providers are expected to be performing at or above for assessments performed during CY 2002
8 Number (%) of Providers At or Above Threshold Level (CY 2002): number of providers at or above threshold level based on vaccination coverage levels from assessments performed during CY 2002