Detailed Information on the
State Children's Health Insurance Program Assessment

Work with states to develop goals for measuring the impact of SCHIP on targeted low-income children through the annual reporting process.

Work with states to develop long-term goals and implement a core set of national performance measures to evaluate the quality of care received by low-income children.

Establishing a methodology to measure improper payments, including producing error rates.

Implement a pilot project to measure SCHIP improper payments and calculate error rates.

Measure: Decrease the Number of Uninsured Children by Working with States to Enroll Children in SCHIP

Explanation:Increase enrollment from baseline.

Measure: Improve Health Care Quality Across the State Children's Health Insurance Program (SCHIP)

Explanation:Target:Collect baseline data on seven core SCHIP performance measures

Is the program purpose clear?

Explanation: The purpose of SCHIP is clearly described in Title XXI of the Social Security Act (SSA); provide funds to States to initiate and expand health care coverage to uninsured low-income children in conjunction with other third party insurers.

Evidence: The Balanced Budget Act of 1997 created the State Children's Health Insurance Program (SCHIP) and provided new funds for states to cover uninsured children. This program represents the largest single expansion of health insurance coverage for children in more than 30 years and aims to improve the quality of life for millions of vulnerable children less than 19 years of age. Under Title XXI of the Social Security Act, states were given the option to set up a separate child health program, expand Medicaid coverage, or have a combination of both a separate child health program and a Medicaid expansion.

Does the program address a specific interest, problem or need?

Explanation: SCHIP addresses the need for health insurance coverage by uninsured, low-income children under the age of 19 with family incomes between Medicaid income levels and 200 percent (and above) of the Federal Poverty Level (FPL). In 2001, the Census Bureau's Current Population Survey (CPS) estimated that the number of uninsured low-income children (defined as under 200% of the FPL) was 5.7 million. Title XXI also extended coverage to uninsured parents whose children are eligible for SCHIP. There is evidence that enrolling parents under 1115 demonstrations and HIFA waivers promotes the enrollment and retention of children in SCHIP and increases utilization of services (see section IV). States may use Title XXI funds to insure parents and other adults, but covering children must remain the highest priority. States cannot cap enrollment of children or institute waiting lists; the priority must be on children over adults. States must ensure that SCHIP funds are available for children over the life of a demonstration that includes parents or other adults.

Evidence: By September 1999, all States and jurisdictions had approved SCHIP plans. Currently, 19 States have separate child health programs, 15 States and D.C. expanded Medicaid coverage, and 16 States have a combination of both programs. States continue to shape their programs through SCHIP state plan amendments. As of April 2002, there have been 155 amendments to SCHIP plans and 12 states have approved section 1115 SCHIP demonstrations to enroll even more children and families. Recently, seven HIFA waivers also were approved (AZ, CA, NM, IL, CO, NJ, and OR) using unspent SCHIP funds. Coverage is now available for children whose income is 200 percent of the Federal poverty level (FPL) or higher in 38 states and the District of Columbia. Prior to this legislation, only six states had income eligibility levels at or above 200 percent for infants only.

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: SCHIP provides an enhanced match rate on health coverage expenditures for uninsured low income children. The enhanced match rate has provided incentives to States and jurisdictions to expand coverage above Medicaid levels. The implementation of SCHIP has increased children's coverage and access to health care to a much greater extent than Medicaid alone. For example, in separate child health programs, SCHIP is not an entitlement, which many States have cited as a determining factor in expanding coverage for children. For States with Medicaid expansion SCHIP programs, the enhanced match has served as an incentive to expand coverage. Apart from the implementation of SCHIP programs, SCHIP has had a positive effect on state Medicaid programs. States have reported that many of the children applying for SCHIP are actually eligible for Medicaid and are enrolled in Medicaid. Also, the outreach and simplification efforts started in SCHIP have "spilled over" to Medicaid and resulted in significant improvements. In addition, many states are implementing premium assistance or employer sponsored insurance (ESI) programs. In ESI programs, the states pay all or part of premiums for group health insurance coverage of an eligible child or children, and employers often pay part of the premium. There currently are 7 states with approved premium assistance programs in SCHIP: Maryland, Massachusetts, Mississippi, New Jersey, Virginia, Wisconsin, and Wyoming. States may also apply for family coverage 1115 waivers under SCHIP, which allows them to purchase coverage for the entire family if it is cost effective. The states with family coverage waivers are Maryland, Massachusetts, Virginia, and Wisconsin.

Evidence: SCHIP enrollment figures show a continued and consistent rise in the numbers of children ever enrolled in SCHIP. In fiscal year (FY) 2002, 5.3 million children were ever enrolled in SCHIP, which is an increase of 700,000 children, or 15 percent, over the 4.6 million children ever enrolled in FY 2001. The 5.3 million children ever enrolled in FY 2002 is more than 2.5 times as many children ever enrolled in FY 1999 and more than four times as many children ever enrolled in calendar year 1998. In comparison to Medicaid, SCHIP has allowed States greater flexibility to change or vary premiums, benefit packages, and delivery systems, as well as subsidizing employer sponsored insurance (ESI) programs. Also refer to Section I, Question #2. SCHIP 1115 demonstrations and HIFA waivers also provide States with additional flexibilities in administering their SCHIP programs. Medicaid data show that enrollment was slow to steady in the early 1990s prior to SCHIP, but began to increase in the late 1990s with the inception of SCHIP.

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: Although SCHIP was designed to focus on the health needs of children, Medicaid and SCHIP have extremely similar functions: To provide health care insurance coverage to low-income people. Currently, many children under age 19 whose family incomes are at or below 100% of the FPL are covered under Medicaid. Prior to Title XXI, States already had the option to increase coverage levels for children under Medicaid or under State-only programs. Health insurance coverage for children has been and could be further expanded under Medicaid. As highlighted in questions 2 and 3 above, SCHIP has had a positive impact on Medicaid eligibility and enrollment; more states now cover children to higher income levels in Medicaid and SCHIP. Medicaid enrollment has increased, and correspondingly the number of uninsured children has decreased since the inception of SCHIP. SCHIP has given States more flexibility to tailor their children's health insurance programs to individual State needs than under Medicaid. Screen and enroll, and crowd out provisions included in the SCHIP regulation also have ensured that eligibility levels and coverage provided through SCHIP funds is not duplicative of Medicaid or private insurance.

Evidence: Refer to section 457.805 of the SCHIP regulation for crowd out provisions and section 457.80(c) for regulatory language on SCHIP coordination with other health insurance coverage. States monitor and report on crowd out to CMS in their annual reports. SCHIP annual reports can be found on the CMS website. The primary method used by states in FY 2001 for preventing crowd out was the imposition of a period during which the applicant must be uninsured prior to enrollment in SCHIP. Thirty-three states (67 percent) reported using periods of uninsurance to prevent crowd out in at least a part of their SCHIP program. Reported periods of uninsurance imposed by states ranged from 1 to 12 months, with 3 and 6 months cited as the most common periods. In addition, a report issued by the Urban Institute in June 2001, 'Has the Jury Reached a Verdict? States' Early Experiences with Crowd Out under SCHIP,' found that states did not have a high incidence of crowd out. A copy of this report can be found on the Urban Institute website. Also refer to Section I, Question #2.

Is the program optimally designed to address the interest, problem or need?

Explanation: The SCHIP formula allocates funds based on each State's uninsured and low-income populations of children as measured by the Current Population Survey (CPS). The allotment formula is designed to concentrate funds in States with the most uninsured children. In addition, it caps Federal liability and gives states flexibility to design their programs and expand coverage. The allotments also serve as a balance to state flexibility in that states are at risk for their choices in designing and expanding coverage. Since the inception of SCHIP in 1997, however, many States have come to rely on multiple years of funding to cover current year program costs. While the redistribution of unspent funds helps States that spend their yearly allotments, States are not guaranteed a set amount of funding and cannot depend on receiving these funds each year. In addition, some States that have expanded to similar coverage levels have large unobligated balances while other States spend most or all of their allotted funds. Currently, the Administration and Congress are considering several proposals that would alter how unspent SCHIP funds are redistributed.

Evidence: Refer to sections 2104(b) (description of the SCHIP formula) and 2104(f) (description of the reallotment process) of the Social Security Act. See the Census Bureau website for the report "The Characteristics of Persons Reporting State Children's Health Insurance Program Coverage in the March 2001 Current Population Survey." The authors point out some of the problems with using the CPS to measure the number of uninsured children, especially in smaller States, in part due to the survey's small sample size for making individual State estimates. Congress specifically has appropriated additional funds to continue to improve both the health insurance questions and sample sizes used in the CPS (See section 2109(b) of the Social Security Act).

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Title XXI states that the main long-term goal of SCHIP is to expand health assistance to uninsured, low-income children. To this end, CMS has a long term SCHIP GPRA goal to increase the number of children enrolled in Medicaid or SCHIP. When the State Children's Health Insurance Program began in 1997, CMS implemented an enrollment goal to enroll five million children by FY 2005. In order to quantify this objective, CMS set annual GPRA targets for FYs 2000 through 2002 to enroll at least one million new children in SCHIP and Medicaid per year. CMS is changing the targets for FY 2003 and 2004 to increase enrollment by five percent over the previous year. This change was made because the program has exceeded the annual GPRA targets for FYs 2000 - 2002 and because states are facing fiscal challenges that may affect program outreach and enrollment, which makes forecasting enrollment difficult. In future years, the ability to achieve this new goal may be impacted by the fiscal situation in the States, increases in the uninsured rate as a result of changes to the U.S. economy, and changes to estimates of the uninsured due to changes in the CPS. In FY03, CMS began developing a GPRA goal to improve health care quality across Medicaid and SCHIP through the Performance Measurement Partnership Project (PMPP). The purpose of this goal is to work with States to establish a core set of quality performance measures that States will report on annually. When fully implemented, these core measures/goals will demonstrate the progress toward the long-term goal of improving health care quality. In 2003, states will be required to report to CMS on these core measures in their annual reports, to develop baselines. However, the program cannot receive full credit until both baselines and long-term targets for the seven SCHIP core performance measures have been developed. HHS should also develop specific and ambitious long-term outcome goals with baselines and targets for SCHIP for the FY06 budget beyond increasing enrollment. Changes in this score will occur only when there is significant evidence to demonstrate results in these areas.

Evidence: Please reference the FY 2004 Annual Performance Plan and Report: 1) Improve Health Care Quality Across Medicaid and the State Children's Health Insurance Program, FY 2004 APP, p. VI-65; 2) Decrease the Number of Uninsured Children by Working with States to Implement SCHIP and by Enrolling Children in Medicaid, FY 2004 APP, p. VI-69. PMPP performance measure examples include: 1) number of well-child visits; 2) access to primary care services; 3) quality of diabetes care; 4) timeliness of prenatal care. CMS will send a request to states in September, 2003 to submit data on the PMPP performance measures in their 2003 annual report.

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: CMS collects performance data from each state. The SCHIP statute requires all states to describe their strategic objectives, performance goals, and performance measures in their state plans. States report to CMS annually on the progress of their performance via annual reports including their progress towards reducing the number of uninsured children in their annual reports. By statute, state annual reports are due to the Secretary by January 1 following the end of the fiscal year. In addition to increasing the number of children enrolled in Medicaid and SCHIP, States have expanded SCHIP eligibility levels. Thirty-eight States and the District of Columbia now have SCHIP income eligibility thresholds of 200% or more of the federal poverty level. Only three states had income eligibility levels this high for children in Medicaid prior to the enactment of the SCHIP program.

Evidence: Refer to sections 2107 and 2108 of the Social Security Act. State annual reports can be found on the CMS website.

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: SCHIP regulations include reporting requirements for States on their individual progress towards meeting strategic and performance goals. These goals are outlined in State plans and reported in annual reports. Many States have set performance goals related to quality and satisfaction of care, and enrollment goals. Information from state plans and annual reports in July 2001 indicated that only 5 States do not use any of the Health Plan Employer Data and Information Set (HEDIS) measures. All other States use all or part of the HEDIS set of measures. Most States collect data on immunizations and well child visits. In addition, States submit descriptions of progress towards enrollment goals in annual reports and must also submit quarterly and annual enrollment data. States also are required to have a plan for outreach and describe their progress in the annual enrollment reports.

Evidence: Refer to the SCHIP regulation, section 457.740(a) for enrollment data requirements and section 457.750 for annual report requirements. SCHIP regulations include reporting requirements for States on their individual progress towards meeting strategic and performance goals, which are reported in the annual reports. CMS reviewed the FY 2001 annual enrollment reports and summarized State outreach efforts as largely successful. States generally employ a variety of outreach methods. In FY 2001, many states described a multi-level approach to outreach, combining broad activities targeting a large audience (such as mass media or mass distribution of SCHIP informational materials) with more targeted, grassroots efforts (such as partnerships with community-based organizations). Mass media strategies ranged from short-term targeted advertising, such as Back-to-School campaigns, to ongoing, extensive campaigns using television, radio, newspaper, billboards and public transit advertisements. Involvement of local grassroots community-based organizations is commonplace in most states, in addition to partnerships with health departments, WIC clinics, Head Start programs, and healthcare providers. See the CMS website for further information on State outreach efforts.

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: States are required to describe in their State plans the procedures they use to accomplish coordination of SCHIP with other public and private health insurance programs including Medicaid and Title V. CMS also works with other agencies to further the goals of SCHIP. CMS and HRSA have a Memorandum of Understanding to ensure effective collaboration and coordination of SCHIP activities, particularly in the area of outreach. Multiple components of HHS and OMB review all State plan amendments, waivers and policy documents. States are required to screen children for both Medicaid and SCHIP eligibility and enroll children in the program for which they are found eligible. State screen and enroll procedures must be included in SCHIP State plans. A report by OIG in February 2001 found that children in the States they surveyed, children were being appropriately enrolled in the programs for which they were eligible.

Evidence: Refer to §457.80(c) of the SCHIP regulation, which describes SCHIP requirements for program coordination. Also see OIG report "Ensuring Medicaid Eligibles are not Enrolled in SCHIP, February 2001" on the HHS OIG website.

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: Every three years, the HHS Office of the Inspector General (OIG) is required to review SCHIP's progress toward reducing the number of low-income uninsured children and properly enrolling Medicaid-eligible children in Medicaid. The General Accounting Office (GAO) is required to monitor the OIG's reports. OIG has issued two reports, one on screen and enroll procedures and the other on the annual evaluations submitted by states. OIG found no problems with State's screen and enroll procedures and CMS concurred. However, CMS will continue to monitor this issue and continue to work with the states to improve screen and enroll processes. On the annual evaluations, OIG recommended that CMS develop a core set of measures and improve the evaluation report framework. CMS, the National Academy for State Health Policy (NASHP), and the states collaborated on a new and improved framework that the states used for the FY 2002 annual reports. CMS, NASHP, and the states are currently working on a web-based annual report template. OIG is currently in discussions with CMS on two future studies of SCHIP. One study will revisit the screen and enroll issue and the other study will assess state progress towards reducing the number of uninsured children as measured by states in the strategic objectives sections of their the annual reports.

Evidence: Refer to Section 2108(d)(1) of the Social Security Act. The three OIG reports, "Assessment of State Evaluation Reports, February 2001", "State Children's Health Insurance Program (SCHIP) Renewal Process, September 2002", and "Ensuring Medicaid Eligibles are not Enrolled in SCHIP, February 2001" can be found on the HHS OIG website. The GAO report, "Children's Health Insurance: Inspector General Reviews Should be Expanded to Further Inform the Congress, March 2002" can be found on the GAO website. See evidence document for study websites.

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: This does not apply since SCHIP funds are alloted to states. The allotment is prescribed by statute and the amount of the allotment cannot be changed in response to program performance. However, each state acts as its own administrative agent and the allotments serve as a balance to state flexibility in that states are at risk for choices in designing and expanding coverage. States must align budgets and goals in order to ensure that the capped SCHIP allotment will cover the costs of the program.

Evidence: Section 2104 of the Social Security Act describes the allotment and reallotment process.

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: Title XXI requires all States to describe their strategic objectives and measures, but there are no consistent measures across all States. The SCHIP regulations require a core set of performance measures and CMS is currently working with the National Academy for State Health Policy to develop this core set. This collaboration is referred to as the Performance Measurement Partnership Project, which will result in a single set of performance measures that will be required of all States. CMS is currently working with the states to develop the technical specifications for the measures that have been selected. A "Dear State Health Official" letter requesting some of this information will be sent to the states in July 2003. Also, CMS will convene a meeting in September 2003 with the states to finalize the specifications for the core set of performance measures. The plan is for States to begin reporting on these measures beginning in their FY 2003 Annual Reports. By statute, state annual reports are due to the Secretary by January 1 following the end of the fiscal year.

Evidence: Section 457.710 of SCHIP regulations refers to the requirement that a core set of performance measures be established for SCHIP. In 2003, CMS is requiring states to collect data on the seven PMPP performance measures and report back to CMS.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Title XXI requires all states to describe their strategic objectives, performance goals, and performance measures in their state plans. States report to CMS annually on the progress of their performance via annual reports including their progress towards reducing the number of uninsured children in their annual reports. In this area, CMS's role is more oversight than operational. Since SCHIP is an insurance program that is managed by a federal/State partnership, the federal government cannot penalize or reward States for how their programs perform unless improper payments are made. States have discretion in setting capitation rates, choosing providers, etc. CMS regularly monitors enrollment growth, enrollment simplification, crowd-out, and other trends to assure that States continue to reach uninsured children. CMS has found that States are making progress in enrolling more children into SCHIP through better outreach and enrollment simplification efforts. CMS gives States feedback on their programs, discusses issues with Regional Office

Evidence: CMS collects performance data through the annual reports, on-site monitoring visit reports (conducted once every two years), and enrollment data (quarterly and annually). CMS, using information obtained from key program partners (the States), is updating and improving the framework used by States to submit their annual reports. In FY 2002, CMS changed the annual report template in response to information and feedback collected from the States. The new annual report template enables the Division of State Children's Health to more efficiently and accurately collect information from the states. The information from these annual reports is then summarized into a comprehensive annual report (which is currently under review). For FY 2003, CMS will provide an electronic form for the states to submit the annual report online, via the web. The new web-based form will further improve the efficiency of the process and the quality of the data submitted. staff, and participates in monthly calls with the SCHIP Technical Advisory Group (TAG), which consists of State Medicaid directors and HHS staff. The SCHIP regulations require a core set of performance measures and CMS is currently working with the National Academy for State Health Policy to finalize the specifications for this core set. This collaboration is referred to as the Performance Measurement Partnership Project, which will result in a single set of performance measures that will be required of all states. CMS is currently in discussions with its contractor, Mathematica Policy Research, to study access and utilization in SCHIP. Allotments are prescribed by statute which means that payments to states will not be affected by state performance as measured by the core set. As CMS collects more extensive performance information from the States, they will be able to utilize baseline data to set more extensive performance goals in the future. More information on this change may be found in Section II, question 5.

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: Without more extensive GPRA and annual performance measures, CMS cannot hold either its managers or the States accountable for cost, schedule, and performance results related to the SCHIP program. CMS staff, however, do follow statutory requirements, such as reviewing State plans and State plan amendments in 90 days. In addition, with Regional Office staff, CMS monitors State financial data to help assure that States are conducting their programs with fiscal integrity. Each State provides projected expenditures, annual budgets, and reports actual expenditures on a quarterly basis. CMS also assesses State budgets as part of all waiver proposals to assure that adequate funds are available to support the state's SCHIP children throughout the life of the demonstration. Since their SCHIP allotments are capped, states do have an incentive to manage their programs' cost, schedule, and performance. States that do not manage their programs well are more likely to exhaust their allotments and not be able to fully fund their programs. Currently, the Administrator of CMS has a performance-based contract that is aligned with some of the performance goals of the program. Other CMS/SCHIP managers are evaluated based on performance contracts that include more process/output measures. In Fall, 2003 the SCHIP Division Director is scheduled to have new performance-based contract that is more closely linked to the program goals.

Evidence: Refer the SCHIP regulation, section 457.740(a) for enrollment data requirements and section 457.750 for annual report requirements.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Title XXI authorizes and appropriates the allotment amounts for each fiscal year from FY??1998 - FY??2007. By issuing grant award notices to the States and territories, CMS obligates all SCHIP funds by the end of the first fiscal year so that States have access to this funding for the entire three years in which it is available. During the course of the fiscal year, CMS issues grant awards based upon each State's request up to each State's allotment for that particular year. Through the reallotment process, States may also receive funds that have been redistributed from other States that could not spend all of their allotments. Even though HHS obligates on a timely basis all of the funds to the States, many States are carrying large unobligated balances due to the lag in enrollment associated with implementing SCHIP and the inefficiencies with the SCHIP allotment formula that results in some States receiving excess funds in relation to the number of low-income uninsured children in their States. Changes to the Current Population Survey should help address this issue.

Evidence: For each year of the SCHIP program the balance in the appropriation for Title XXI will show a zero balance indicating clearly that all funds have been obligated.

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: This does not apply since SCHIP funds are allotted to States, which determine their own contracts.

Evidence:  

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: This does not apply since SCHIP funds are allotted to States, and States must keep administrative costs under 10% of their total program costs. The enabling legislation provided for State and local, but not Federal CMS administrative costs to be funded from the amounts appropriated in the BBA. State and local administrative costs are statutorily capped. The States report on these administrative costs quarterly. CMS does not budget separately for Federal administrative costs, either in terms of dollars or FTE employment.

Evidence: Refer to section 2105(c)(2)(A) of the Title XXI statute showing the 10% cap that applies to State and Local Administration.

Does the program use strong financial management practices?

Explanation: The Federal Financial Management Improvement Act (FFMIA) of 1999 requires that Federal programs must assess improper payments rates and do risk assessments. In the past, CMS has not calculated error rates for SCHIP. Currently, CMS is working with the States to develop an SCHIP error rate through the Payment Accuracy Measurement (PAM) project. In FY 2004, CMS is encouraging up to twenty five states to volunteer to pilot test the CMS PAM Model in both their Title XIX Medicaid and Title XXI SCHIP programs. At the conclusion of the year, the final specifications for the CMS PAM Model will be produced in anticipation of nationwide implementation. As CMS implements the PAM Model, they will be able to track and lower improper payment rates in the future. Additionally, CMSO's Division of Financial Management conducts ongoing risk assessments at the regional offices in order to pinpoint areas of risk. The CMS reviews are periodically audited/used by the HHS OIG, the GAO, and audits conducted annually under the Single Audit Act. CMS has a structured Financial Management (FM) workplan process for SCHIP, which is updated annually. The FM workplan incorporates risk analyses, FM reviews, structured planning and FM oversight of the SCHIP program.

Evidence: CMS is soliciting States to participate in the PAM project. CMS will issue PAM grants by the end of the fiscal year to States who elect to participate in the project. See attached draft version of the CMS PAM Model which includes applications to SCHIP. The Financial Management workplan includes front end financial management on Administrative program management and Services program development (e.g., reviews of cost allocation plans, administrative claiming plans, prior approval of contracts, technical assistance), Ongoing FM Oversight/Enforcement (e.g., focused FM review on high risk areas, audit liaison, deferrals and disallowances, and data gathering and analysis), and finally Quarterly Reviews related to states Budget and Expenditure reports. One of the primary emphases in these activities is the focused FM reviews, in which risk analysis on vulnerable areas is done and specific areas of reviews in each RO are identified and implemented. With the resource constraints, these activities are conducted both with respect to the SCHIP and Medicaid programs. As the year progresses, the ROs report on their progress.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Although CMS monitors states through State plan amendments, monitoring visits, data and financial reviews, they are just beginning to take adequate steps to address Federal Financial Management Improvement Act (FFMIA) requirements for SCHIP. In response to recent GAO reviews and recommendations, CMS has begun to institute a structured Financial Management (FM) workplan process for SCHIP, which incorporates risk analyses, FM reviews, structured planning and FM oversight of the SCHIP program. In order to comply with Federal Financial Management Improvement Act (FFMIA) requirements, CMS is working with the States to develop a SCHIP-error rate through the Payment Accuracy Measurement (PAM) project and will begin a pilot demonstration in FY 2004.

Evidence: See section III, question 6.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: States must submit State plan amendments for all significant program changes. In order to ensure that States conduct their SCHIP programs as they described in their state plans, CMS conducts on-site monitoring visits, works with regional offices on day-to-day monitoring activities, and requires annual reports and quarterly data submission. By monitoring financial and enrollment data, CMS determines if States are utilizing their allotments to meet the goals of Title XXI. In addition, Title XXI authorizes the reallocation of funds from states that do not use them to states that need funds.

Evidence: Refer to sections 457.40(a) and 457.720 of the SCHIP regulation for a description of monitoring activities.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: CMS places the following materials for each State on the CMS SCHIP web site: State plans and amendments, annual reports, evaluations, enrollment information, and national SCHIP evaluations conducted by independent contractors. Once implemented, demonstrations are monitored through review of quarterly and annual reports, regular CMS/State communication, and site visits. CMS has funded several independent evaluations by private contractors to assess the impact of certain approved demonstrations on service delivery systems, costs, and quality of care. States with approved HIFA waivers must include an evaluation component. CMS will award a RFP contract this fiscal year both for an evaluation of the recently approved and future HIFA waivers.

Evidence: Refer to CMS SCHIP web site: www.cms.hhs.gov/schip CMS also is preparing the first annual summary of State annual reports, which will be placed on the CMS SCHIP web site.

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: There were 5.3 million children enrolled in SCHIP in FY 2002. This was an increase of 700,000 children (or 15 percent) over FY 2001. A recent CDC study also found that children are significantly more likely to be insured now than in 1997 when SCHIP was enacted. In terms of SCHIP 1115 demonstrations for parent coverage, recent studies have found that States with parent coverage are more likely to enroll children in SCHIP and Medicaid and utilize more health care services. Also, many States have observed that enrollment of parents promotes enrollment and retention of children, as well as utilization of services. CMS's enrollment GPRA goal demonstrates the annual progress towards the long-term goal of decreasing the number of uninsured by enrolling children in SCHIP and Medicaid. In FY 2003, CMS also began developing a GPRA goal to improve health care quality across Medicaid and SCHIP through the Performance Measurement Partnership Project (PMPP). The purpose of this goal is to work with States to establish a core set of quality performance measures that States will report on annually. When fully implemented, these core measures/goals will demonstrate the progress toward the long-term goal of improving health care quality. As noted in Section II Question 1, over the past year, HHS will require states to report on the seven SCHIP core performance measures to develop baselines. HHS also should develop specific and ambitious long-term outcome goals with baselines and targets for SCHIP for the FY06 budget beyond increasing enrollment. Change in this score will occur only when there is significant evidence to demonstrate results in these two areas.

Evidence: A recent Urban Institute presentation reported that in States that have expanded coverage for parents under Medicaid 81 percent of eligible children participate in Medicaid compared to only 57 percent of children in States without family-based coverage programs. A recent CDC study also found that the percent of children (17 and under) without health insurance declined from 13.9 percent in 1997 to 10.1 percent between January and September 2002. During this period, reliance on public programs for coverage was fairly constant between 1997 and 2000 at about 21 percent, but then rose to 23.4 percent in 2001 and jumped to 27.2 percent in 2002. As public coverage rose, the percent of children covered by private plans dropped from 67.1 percent in 2001 to 64.2 percent 2002. This report can be viewed on the CDC website at ttp://www.cdc.gov/nchs/products/pubs/pubd/hestats/insurance.htm. See the HHS website (odphp.osophs.dhhs.gov/pubs/HP2000/2010.htm) for information on Healthy People 2010. Lastly, CMS has a contract with Mathematica Policy Research, Inc. (MPR), for a number of SCHIP evaluation activities. MPR is working on a report that will describe the changes in the number of uninsured children in the U.S. relative to implementation of SCHIP and recent trends in Medicaid enrollment using data from the Current Population Survey (CPS). MPR's preliminary analysis suggests that at least half of the decline in the CPS number of uninsured children may have been due to SCHIP, with traditional Medicaid growth accounting for another 10 to 15 percent. Please reference the FY 2004 Annual Performance Plan and Report: 1) Improve Health Care Quality Across Medicaid and the State Children's Health Insurance Program, FY 2004 APP, p. VI-65; 2) Decrease the Number of Uninsured Children by Working with States to Implement SCHIP and by Enrolling Children in Medicaid, FY 2004 APP, p. VI-69.

Does the program (including program partners) achieve its annual performance goals?

Explanation: CMS is making progress towards implementing core performance measures. CMS currently is working with the States and the National Academy of State Health Policy on developing a core set of performance measures that rely heavily on he Health Plan Employer Data and Information Set (HEDIS) measures but this process is in the early stages, in part because the SCHIP program is just 5 years old. HEDIS contains well established quality health measures and is administered by the National Council for Quality Assurance. New GPRA goals and annual core performance measures will further help improve the SCHIP program. Status Update: CMS is currently working with the states to develop the technical specifications for the measures that have been selected. A "Dear State Health Official" letter requesting some of this information will be sent to the states in July 2003. Also, CMSwill convene a meeting in September 2003 with the states to finalize the specifications for the core set of performance measures. The plan is for States to begin reporting on these measures beginning in their FY 2003 Annual Reports. By statute, state annual reports are due to the Secretary by January 1 following the end of the fiscal year.

Evidence: See evidence provided in section II, question 2. The "Status Report for the Performance Measurement Partnership Project (PMPP)" discusses how CMS and the PMPP workgroup have developed a list of 9 core performance measures related to improving the quality of care for children. Six of the nine measures already are included in HEDIS. Over the next several years, the PMPP workgroup will continue to work on implementing these measures. Refer to CMS for additional information on the PMPP workgroup Also, please reference the FY 2004 Annual Performance Plan and Report. CMS monitors: 1) Improve Health Care Quality Across Medicaid and the State Children's Health Insurance Program, FY 2004 APP, p. VI-65; 2) Decrease the Number of Uninsured Children by Working with States to Implement SCHIP and by Enrolling Children in Medicaid, FY 2004 APP, p. VI-69.

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: This does not apply since SCHIP funds are allotted to States. On a federal level, CMS does not have the authority to require State programs to be cost effective. As previously stated, SCHIP does not have long-term or annual performance goals that focus on cost effectiveness. Individual States, however, can assess cost-effectiveness in their own programs but these goals are not linked to Federal program goals.

Evidence: Refer to the FFMIA of 1999.

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: SCHIP enrollment has increased steadily, as has Medicaid enrollment. SCHIP has also had positive effects on Medicaid. States have simplified Medicaid enrollment and many children have been enrolled in Medicaid as a direct result of SCHIP. SCHIP has changed the perception of the Administration, Congress, states, advocates, and families, of public coverage for children. SCHIP provided States with a unique opportunity to model public coverage after the private sector, allowing more flexibility on benefits and cost sharing. This was particularly important to States as they covered children with higher family incomes. Most new proposals for government expansions of coverage are modeled on SCHIP because of the strong consensus that SCHIP is a successful program.

Evidence: Refer to SCHIP Annual Enrollment Reports on the CMS webpage. For an example of changes to Medicaid, refer to CMS's "Continuing the Progress" report on the CMS website.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: A number of independent evaluations have found that SCHIP is effective in increasing health insurance coverage for low-income children. Data from both the 2000 CPS and from the CDC in 2001 show a decrease in the number of uninsured children (under the age of 19) compared to previous years. There still, however, are needed improvements in the program. Future program improvements need to continue to emphasize decreasing the rate of the uninsured and increasing access but in addition focus on type and quality of services. The CMS evaluation describes program design and implementation in the states, including program features and outreach strategies that encourage enrollment in SCHIP. The ASPE evaluation found that there is high enrollee satisfaction and positive attitudes toward SCHIP. SCHIP has succeeded in enrolling millions of children and has also helped to increase enrollment in Medicaid, program entry in SCHIP and Medicaid has been streamlined, states continue to improve and tailor outreach strategies, SCHIP offers good access to care, and there continues to be ongoing support for SCHIP. Both the ASPE and CMS evaluations were performed by an independent contractor.

Evidence: Mathematica Policy Research, the National Academy for State Health Policy, the Urban Institute, and HHS' ASPE have evaluated the SCHIP program (See websites for each organization). In addition, the Agency for Healthcare Research and Quality (AHRQ), the David and Lucile Packard Foundation, and the Health Resources and Services Administration (HRSA) currently are funding eight research projects that include SCHIP over the next three years through the Child Health Insurance Research Initiative (CHIRI). These studies seek to uncover which health insurance and delivery features work best for low-income children, particularly minority children and those with special health care needs. CMS and ASPE both contracted with outside organizations for major evaluations of SCHIP for Congressionally-mandated reports (the executive summaries are included in the evidence document).