This report was produced under U.S. Department of Education Contract No. ED-01-CO-0012, Task Order D010, with WestEd. Sharon Horn served as the contracting officer’s representative.

U.S. Department of Education
Rod Paige
Secretary

Office of Innovation and Improvement
Nina S. Rees
Deputy Under Secretary

Michael J. Petrilli
Associate Deputy Under Secretary for Innovation and Improvement

Thomas M. Corwin
Associate Deputy Under Secretary for Policy and Planning

May 2004

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should be: U.S. Department of Education, Office of Innovation and Improvement, Innovations in Education: Creating Strong Supplemental Educational Services Programs, Washington, D.C., 2004.

To order copies of this report,

write to: ED Pubs, Education Publications Center, U.S. Department of Education, P.O. Box 1398, Jessup, MD 20794-1398;
or fax your request to: (301) 470-1244;

or e-mail your request to: edpubs@inet.ed.gov;

or call in your request toll-free: 1-877-433-7827 (1-877-4-ED-PUBS). If 877 service is not yet available in your area, call 1-800-872-5327 (1-800-USA-LEARN). Those who use a telecommunications device for the deaf (TDD) or a teletypewriter (TTY), should call 1-877-576-7734;

or order online at: http://www.edpubs.org/.

This report is also available on the Department’s Web site at: http://www.ed.gov/nclb/choice.

On request, this publication is available in alternate formats, such as Braille, large print, or computer diskette. For more
information, please contact the Department’s Alternate Format Center at (202) 260-9895 or (202) 205-8113.

Figure 1. NCLB School Improvement Timeline

Figure 2. State and District Roles

Figure 3. Demographics of Five Profiled School Districts

Figure 4. Key Actions to Implement Supplemental Educational Services

Figure 5. SES Start-Up Steps

Figure 6. Agenda from Los Angeles Meeting with Service Provider

Figure 7. Resources to Help Community Organizations Become SES Providers

Figure 8. San Diego Enrollment Form

Figure 9. Questions for Providers

Figure 10. Los Angeles Flyer

Figure 11. Toledo School Site Instructions

Figure 12. Los Angeles Unified School District Student Learning Plan (Sample)

Figure 13. Forsyth Teacher-Tutor Data Collection Sheet

Figure 14. Forsyth Biweekly SES Progress Report

Figure 15. Rochester Parent Survey

Figure 16. Study Scope and Guiding Questions

I am pleased to introduce the second publication in the Innovations in Education series: Creating Strong Supplemental Educational Services Programs. This series, to be released over the next six months, identifies concrete, real-world examples of innovations flourishing throughout this great land, in six important areas: public school choice, supplemental educational services, charter schools, magnet schools, alternative teacher certification, and school leadership.

Although the term "supplemental educational services" is enjoying newfound prominence, its meaning is as old as education itself: tutoring. This important provision of the No Child Left Behind Act provides eligible low-income parents with the same opportunities more affluent parents have long enjoyed: the chance to engage a highly skilled tutor, or access other forms of academic enrichment, to help their child catch up if they have fallen behind. For school districts, this extra help for their neediest students can be an important complement to ongoing school improvement efforts.

But as we have learned in the first two years of this historic law, successfully setting up a supplemental educational services program takes a lot of work and foresight. States play a key role by approving and monitoring the "providers"—including nonprofit organizations, faith-based groups, for-profit companies, collaboratives of teachers, and school districts—that may deliver the tutoring. Parental choice is central—the statute purposely sets up a marketplace so that parents can find a provider that works best for their child's needs. And in between parents and providers is the school district, ideally serving as a fair broker, contracting with and paying providers, informing parents, and making sure the system is working smoothly. Of course, doing all of that is easier said than done.

This report was developed to give district leaders some guidance as they implement supplemental services. In doing so, it draws on examples from five diverse districts across the country whose implementation experiences yield some common themes and lessons that might be helpful to others working on supplemental services. For instance, successful districts embraced the spirit of supplemental services by setting a positive tone about the importance of these provisions, planning for their implementation, and staffing the program adequately. They built strong relationships with providers, helped them access school facilities, and created a fair contracting system. They used multiple methods to inform parents of their options and helped them make the best choice for their children. And they established clear student learning goals and ensured that providers were supplying frequent progress reports to parents and teachers.

Certainly, none of these districts is doing everything perfectly. This reform is still only a few years old; implementation will improve as time goes on. My hope is that this booklet can help to accelerate the learning curve as districts across the nation learn from these "early adapters."

As with public school choice, one message of this publication is that "it can be done." With effective district leadership and a consistent focus on what this is all about-ensuring that none of our children is left behind-I am confident that these supplemental services programs will be a great success.

Rod Paige, U.S. Secretary of Education
May 2004

When Deborah George's eight-year-old daughter Melissa* entered fourth grade, she was still struggling with reading. She had just started a new school and felt anxious when she had to read aloud in front of her classmates. "I hate reading," she told her mother.

Deborah tried to help her daughter at home, but it was a challenge. When she was able to convince Melissa to read aloud, Deborah discovered that she was guessing at words rather than sounding them out and was not understanding very much of what she was reading. Her poor reading skills were causing Melissa to fall behind in other subjects as well.

Frustrated in her own efforts and anxious to find help for Melissa, Deborah asked her daughter's teacher to recommend a private tutor. But she quickly found out that the cost of individual tutoring was well beyond her financial means.

A short time later, Deborah received what she calls "a remarkable letter" from the Toledo Public Schools telling her that Melissa was eligible to receive free tutoring. The letter explained that this opportunity was a result of the No Child Left Behind Act of 2001. It also said she could choose any service provider she wanted from a list that had been approved by the Ohio Department of Education. Included with the letter was a brochure identifying the providers and giving contact information for each one in case parents wanted more information. Deborah notified the district of her choice and soon thereafter Melissa began spending an hour and a half twice a week with the tutor her mother had chosen.

Six weeks into the tutoring, Deborah enthusiastically described Melissa's progress: "She says she loves to read now, and she even raises her hand to read aloud in class. She's made so much progress in such a short time that it's just amazing. Her teacher noticed her improvement almost immediately. I made a great choice with this tutor: He assessed Melissa's skills and zeroed in on just what she needs, and he gives her practice to do at home. For the first time, she loves doing homework. The most amazing thing is that she has asked us for books for Christmas because she loves to read."

Melissa continued to make progress in reading, and, as a result, her grades in other subjects steadily improved. She passed the fourth grade reading proficiency test on her first try.

Melissa beams confidence and self-esteem when she describes her progress. Asked how the tutoring has helped, she thinks for a minute and then says, "Well, before I couldn't read. Now I can. I think anyone who has a reading problem should get a tutor like Mr. Miller."

Supplemental Educational Services: Giving Parents More Options, Giving Students Extra Help

When it comes to picking up on warning signs that their child is having difficulty in school, parents are pretty intuitive, and they want to act. But how? The common sense notion that some children need more instructional time than others to master the curriculum is supported by research and theory. If all students are to achieve to grade-level standards, every student must receive the specific support that he or she needs as a learner, including extra time with individual attention and precisely focused instruction. Studies show that students who continue to struggle in school without intervention compound their learning losses into a larger deficit that is difficult to remediate. In contrast, carefully tailored learning interventions can yield quite remarkable and swift progress in overcoming learning obstacles, as evidenced in Melissa's experience working with a reading tutor.

Parents who can afford it have commonly obtained this type of intervention for their child by paying for private tutoring or after-school skill-building courses. Low-income parents want the same extra support for their children, but haven't had that option. Now, thanks to No Child Left Behind (NCLB), their options are expanding. NCLB's Supplemental Educational Services (SES) provision gives parents of eligible children the opportunity and the funding to choose a private tutor or other academic support provider to help their child succeed in school.

This focus on parental choice places significant emphasis on parents' knowledge and understanding of their child's education needs. It also demonstrates a confidence in parents' ability to choose the most appropriate academic intervention for their child from among marketplace competitors. An important tenet of the law is that low-income parents should have the same range of options available to parents who can afford to scan the marketplace and select an academic intervention service that meets their child's needs. Under NCLB's SES provision, a key responsibility for states and districts alike is to present eligible parents with as many diverse provider options as possible. The services offered across the country include one-on-one tutoring, small-group prescriptive skill-building, individualized gap assessment and remediation, small-group drill and practice, computer-based assessment and skill-building, interactive e-tutoring on the internet, and internet-based skill-building with direct feedback.¹ The settings in which children participate in SES activities include their own schools or another nearby district school, community centers, faith-based centers, libraries, service providers' centers, computer centers, and their own homes. When parents are given a wide variety of provider options, service providers must compete for their business. This competitive market encourages continuous improvement of program quality and services to students and their families.

The District's Role in SES

States and school districts both play integral roles in designing the scaffolding to support parental choice, but it falls primarily to the local district to bring SES to life for its families. To aid in that effort, this guide shares the early implementation experiences of five districts across the country. Varying in size and setting, each has struggled with the same issue: how to ensure that parents of eligible students can realize the full potential of the choice granted to them by this historic legislation. The districts are not presented as exemplars. Similarly, district artifacts included in the guide (e.g., an SES promotional flyer) are not presented as ideal models but rather as examples. Nor are their implementation strategies proposed as perfect practice. But for the last two years these local educational agencies have been busy figuring out what works for them, what works better, and what doesn't work at all. From their experiences, some common themes emerge and some lessons that might be helpful to others heading down this road—first among them that SES provides an opportunity to bring more partners to support the work of schools and teachers to improve achievement of their lowest-performing students. Their stories are presented in that light.

Getting clear on the SES requirements and, even more, on what districts need to do to implement those requirements is the first challenge. States are responsible for soliciting, screening, and approving providers and for maintaining an updated list based on providers' performance record. Parents get to decide which provider to use, and they receive regular reports from the provider about their child's progress. Independent contractors provide the services (as can the district itself if it has been approved by the state as a provider). In the middle is the district, whose task is to create the conditions and manage the logistics that make it possible for parents to exercise their right to choose a service provider for their children.

Among their "first steps," districts need to establish contracting relationships with service providers and develop a notification and application process for parents of eligible students. Once new structures and processes are initiated, attention can turn to "going deeper," including orchestrating a communications plan that engages more parents, expanding community-based networks to keep parents informed about their options, and building on SES approaches to extend academic intervention opportunities broadly throughout the district. In all SES efforts, districts should stay focused on the goal of ensuring that parents have easy access to as broad a choice of providers as possible in order to find the appropriate support for their child.

Some basics: As shown in figure 1, a school that does not make AYP for two years running is labeled "in need of improvement," a designation that requires its district to offer public school choice to students in that school the next year.² (If districts are unable to offer choice, they are encouraged to offer SES to eligible students instead that first year.³)If the school misses its AYP for a third year, the NCLB's supplemental educational services provision kicks in. If the school continues to miss AYP, moving into corrective action and, then, restructuring, the district must continue offering choice and SES to eligible students.

SES-eligible students are low-income students in those schools; if there are insufficient funds to serve all eligible students, priority goes to the lowest-achieving students. Districts with schools in need of improvement are required to spend an amount equal to 20 percent of their Title I, Part A allocation on a combination of supplemental educational services and any transportation required for choice under NCLB, with a minimum of 5 percent dedicated to SES. Districts do not have to provide transportation for SES. None of the 20 percent can be spent on administrative costs, which must be covered in some other fashion. A district must provide, for each child's services, an amount equivalent to the district's Title I, Part A per-child allocation (the amount of Title I, Part A funds the district receives, divided by the number of poor and other children counted under the federal census Title I formula), unless the actual cost of services is lower. District Title I per-child allocations vary greatly across the country, but generally range from $750 to $1900.⁴ For more information on state and district SES roles, see figure 2. For additional guidance, see the Office of Innovation and Improvement Web site: http://www.ed.gov/about/offices/list/oii/about/choice.html.

Case Study Sites and Methodology

The five districts profiled in this guide are Forsyth County, Ga.; Los Angeles, Calif.; Rochester, N.Y.; San Diego, Calif.; and Toledo, Ohio. Basic statistics about these districts appear in figure 3. For a narrative summary of each district's context and programs, see appendix A.

Figure 3. Demographics of Five Profiles School Districts

These five districts were selected from a larger set of possible sites as part of the benchmarking methodology that underlies this study. Thirty-six districts were identified as potential sites for one of two reasons: they were SES-eligible districts in states that had actively addressed SES and had many approved providers, or they were suggested as districts that had actively addressed SES in the view of state department staff, SES providers, or members of the advisory group. Existing public data and targeted interviews provided preliminary information about these districts that was used to "screen" sites and identify those that appeared to have practices in place in several key operational areas. For example, they had clearly articulated strategic plans for administering SES, outreach and communications strategies for both parents and providers, and explicit contracting and recordkeeping procedures.

This exploratory, descriptive approach is adapted from the four-phase benchmarking process used by the American Productivity and Quality Center (see appendix B for further details). In benchmarking, organizations analyze their own operations and look to promising practice partners for ideas of specific practices that might help them improve. For this study, an advisory group of researchers, providers, and practitioners with experience in supplemental educational services helped guide the focus. Their input, together with an examination of research literature and an analysis of NCLB requirements, led to the study scope (see appendix B).

Descriptions of the districts' practices were collected through two-day site visits that included interviews with providers, principals, and parents, as well as district staff. The districts arranged these interviews and also provided copies of artifacts, such as sample letters, brochures, contracts, lesson plan forms, and so forth. For each district, the study team then summarized in an individual case report the practices and lessons learned; a cross-site analysis organized the findings by topic and revealed common patterns. This guide is adapted from the full research report and also incorporates advice from the U.S. Department of Education's Office of Innovation and Improvement, which, jointly with the Office of Elementary and Secondary Education, oversees implementation of the NCLB supplemental educational services provision. Results from specific district practices, district rationales for what they did, patterns across districts, and common sense, along with the initial framework, led to the themes and suggested actions in this guide.

This descriptive research process suggests promising practices—ways to do things that others have found helpful or lessons they have learned about what not to do—and practical "how-to" guidance. This is not the kind of experimental research that can yield valid causal claims about what works. Readers should judge for themselves the merits of these practices, based on their understanding of why they should work, how they fit the local context, and what happens when they actually try them. Also, readers should understand that these descriptions are not intended to add any requirements beyond what is already in the NCLB statute and regulations.

Organization of the Guide

This guide shares practical ideas from districts around the country that have been learning as they go in the early implementation of SES. It is organized around four action areas and specific key actions in each—some that are first step, some that are going deeper (see figure 4).

If SES is to have a chance of reaching its full potential to empower parents and improve student learning, it cannot be held at a distance, treated as yet another compliance task. Rather, districts must approach SES with open arms, taking advantage of new partnerships and welcoming it for what it can add to the educational experience of their lowest-achieving students. Setting a positive tone, staffing SES strategically, identifying and addressing potential barriers to parent participation, planning rather than waiting for state guidance on key issues-these are essential steps to making SES work. To do otherwise risks confusing miscommunication, missed opportunities, inadvertent redundancies in services to students and in management of the program, and SES being undermined by other priorities-all at the cost of precious resources, student learning most of all. Once the SES program is up and running, districts may choose to adapt or create new roles and structures to more deeply institutionalize it.

First Steps

Set a Positive Tone

The words and actions of district leaders make their priorities known and symbolically announce what matters most to them. As evidence of their commitment to the goals of NCLB, San Diego leaders quickly initiated a district wide NCLB coordinating committee that consists of department directors and other key staff and that meets weekly. The intent of having frequent and face-to-face discussions is to ensure consistent strategies and communication, within the central office and between the district and its schools. The overall message is that the provisions of NCLB, including SES, are good for the district, good for its schools, and, most important, good for San Diego students and their parents, so get on board. This affirmative stance has prompted 9principals and teachers in SES-eligible schools to view the program as an important adjunct to their own strategies for improving student achievement, thus ratcheting up their commitment to communicating with parents about this new opportunity.

A positive view is not always the initial reaction to SES, as educators in Toledo and elsewhere have discovered. Some in Toledo, for example, initially fretted about the funding earmarked by NCLB for SES, seeing it as money taken away from school programs in which they were already invested. Acknowledging the initial frustration, Toledo's Chief Academic Officer says:

"The turning point for us came when we began to see supplemental services as a great way to give extra support to the kids who needed the most help. There should be more help for our neediest kids... Ultimately, [SES] will help our school improvement buildings meet their AYP targets. That's good."

With its revised perspective, Toledo went on to create a vigorous parent outreach strategy that increased the participation of parents by 130 percent.

Staff SES Strategically

District SES programs need to be staffed by individuals who can focus time and attention on getting it going, who understand the goal-ensuring easy access for parents to this new opportunity for their child-and the many operational tasks necessary for achieving the goal.

All five districts sought an administrative home for SES that offered existing expertise, cost-effective operation, and efficient implementation in order to jump-start the program early in the school year while laying the foundation for building a more developed and fully realized SES system. To get things started, both San Diego and Los Angeles made SES the responsibility of the particular department that was already responsible for managing existing extended learning programs throughout the district. Two other districts, Forsyth and Toledo, placed SES under the aegis of their Title I program. Rochester, on the other hand, has initially located SES in its Office of Accountability and Academics because of that office's experience nurturing new initiatives. This way the program can get the extra attention needed to get off to a good start. One benefit is that this same office also houses Rochester's well-developed accountability database and student achievement data system, both of which are likely to be utilized and expanded in tracking SES effectiveness.

The point in all cases is to capitalize on any existing structures that are well suited to overseeing a multifaceted program like SES, which requires everything from effective parent outreach to contract monitoring. At the same time, however, districts must find ways to ensure that SES retains a distinct identity within the district. SES emphasizes parental choice and this important aspect must not get lost or receive short shrift in departments that already have a lot of other responsibilities.

Put a Plan in Place

Each district in the study cited early planning and preparation as central to its ability not only to get SES in operation but also to align SES with strategic goals, maximize the impact on student learning, and avoid unnecessary duplication of effort. Looking back on their own implementation experience, SES administrators in all five districts say the same thing: Anticipate-don't wait for the state. This applies whether talking about waiting for official notification about SES-eligible schools before starting to plan initial implementation or waiting for the state to develop a quality assurance system for providers rather than starting right away to identify and create a district process for collecting relevant data.

For the most part, districts in this study were prepared to begin implementation as soon as they received eligibility data from their state departments of education. For example, anticipating that 104 of its more than 700 schools would be required to offer SES and recognizing the size of the undertaking, Los Angeles began planning in the spring and summer before implementation was actually required. Although few districts will operate an SES program on the scale of Los Angeles, planning procedures and processes in advance of actual implementation is an important consideration for every district offering SES. Having a game plan and knowing how to execute it can help district staff align their responses and actions with district goals. Such a plan supports consistency of the district's message to parents and the public.

Rochester's district leaders began communicating about SES regulations six months in advance of starting the program. Anticipating that many parents would seek information from their children's schools about the free tutoring and academic support available through NCLB, Rochester proactively outlined its SES implementation plan and informed school personnel of the procedures it intended to follow.

The list of SES start-up steps in figure 5 draws from the experiences of these five districts as well as the advice of the Office of Innovation and Improvement. Although the order in which these steps are taken may vary according to local circumstances, this list offers a reasonable map for advanced planning. The later sections of this guide elaborate on many of these steps.

For SES resources that may help in planning and implementation, see appendix C: Resources.

Going Deeper

Review Results and Improve Processes

Once districts are past the first steps of putting an SES program in place, they have a chance to review implementation-including key indicators like the number of students served-and make adjustments. Rochester wrote a three-page memo outlining issues and solutions, and concluded:

"In general the '02-'03 academic year was an attempt by the RCSD to put together an SES program that complied with NCLB. Having succeeded on that front, our interest has turned to strengthening the program to provide the greatest academic gains to the greatest number of students."

Meeting the requirements to comply with the mandates of NCLB is a beginning step. Ensuring that all eligible students receive the services intended for them, and monitoring those services to ensure intended impact, requires a deeper investment of leadership and commitment. Taking up that challenge calls for strategic and imaginative planning, as well as persistence over time.

Identify Barriers to Parent Participation

SES conveys to parents the right to choose a tutor or academic support service for their eligible children. In planning SES implementation, it's important for a district to think and act systemically, not just about how to fit the program into an effective administrative structure, but also how to manage it so as to engage the greatest number of parents and give them access to as broad a range as possible of high-quality SES providers for their children. This means identifying and, to the extent possible, eliminating potential barriers to parents. As an example, many parents-especially working parents-manage complex schedules in getting their children safely to school, arranging after-school care, and then picking their children up to go home. Districts are not required to transport children to and from off-site SES sessions; therefore, the parent of an elementary school student who attends a school-based or center-based aftercare program may be less likely to enroll the child in any SES program that does not provide services at the child's school, the after-school center, or in their home. Depending on the respective locations of the school and an SES provider, even parents of older students may not feel comfortable having their child walk or take a city bus to an off-site tutoring session. Although a district has no mandated responsibility to make sure providers are conveniently available to eligible students, to facilitate broader participation, in the spirit of the law, it could, for example, initiate conversations with providers, inviting them to locate at schools. Where center-based rather than school-based aftercare is the norm, a district might broker similar conversations between providers and popular local aftercare programs, such as a YMCA or Boys and Girls Club-again, for the purpose of working out a system whereby providers could offer services at the center so that parents who need full-time aftercare for their child sign up for SES. In cases where such co-location of services is not possible (e.g., when a provider runs a computer-based program at its own center), a district might be able to do something as simple as providing parents of older students with a public bus schedule or may want to examine its own bus schedule to see if it could be easily adjusted without additional cost to transport some students to SES providers.

Establish New Roles and Structures

After they get SES started, districts have a chance to look at how things are working and think through what roles and structures might strengthen the program. Sometimes, new positions at the school or district level are called for. Rochester, for example, created a new position at targeted improvement schools to reach out to parents and to coordinate academic interventions, including facilitating ongoing exchange of information between providers and classroom teachers.

After meeting all NCLB requirements for parent notification, Forsyth looked for ways to connect more effectively with non-English-speaking parents of SES-eligible students. The answer was a new Transition Center. Coordinated by bilingual staff with credibility in the community, the center provides parents with assistance in school registration, placement testing, and many other areas as they settle into the community. Forsyth credits Center staff for breaking down cultural and language barriers, increasing the involvement of parents, and engaging more of them in enrolling their children in tutoring.

Make SES a Complementary Part of Ongoing Extended Learning Programs

All five districts in this study had existing programs that provided extended learning opportunities for students. They tended to see SES as a "congruent" effort that, in some instances, would enable them to get services to more students and, in other instances, would help them get more services to the neediest students. Beyond deciding where to place SES in the district's organizational structure, as discussed above, districts wanting to make the most of this new student resource will want to address the larger question of how best to capitalize on the full array of extended learning resources, including SES, to meet students' needs.

Rochester, for example, has worked to develop strong linkages to its state-mandated Academic Intervention Services (AIS) system, which provides extra support for students who have not passed, or are considered at risk of not passing, New York's standards-based assessments in key academic areas. Although SES is not a component of AIS in the district organizational structure, the district has created a number of deliberate connections between the two programs. Students can participate both in extended-day services under AIS and in NCLB supplemental services. As mentioned above, a newly created position of AIS specialist at targeted schools helps work out the coordination. In addition, during development of their child's SES plan, parents are encouraged to provide information from past AIS Progress Reports. This information helps providers understand students' distinct skill needs and align their instruction with the indicators being assessed in the AIS system, which reflects state standards. One key purpose of this coordination is to ensure that when students are receiving both AIS and SES services, the interventions are complementary, not conflicting or needlessly repetitive.

In Forsyth, the district's positive experience with an SES provider has prompted it to expand that relationship beyond the NCLB requirements: Because the State of Georgia has sought to end social promotion by mandating the retention of third grade students who do not pass the state's criterion-referenced reading assessment, Forsyth has approached the tutoring program about providing one-on-one instructional assistance to any students at risk of being retained in third grade, irrespective of whether they are eligible for SES

As noted earlier, in some instances, a district may want to encourage and support co-location of SES services with pre-existing extended learning programs and aftercare programs. In addition to eliminating possible transportation barriers to SES for some families, location of multiple services at one site would allow students to more easily take advantage of multiple intervention services as appropriate.

Strengthen Parent Advisory Groups and Partnerships

Districts benefit from engaging parents as real partners in SES implementation, for example, by strengthening formal roles for parents in district decision-making. Many times, districts can expand the participation of existing parent organizations. Los Angeles staff met with school parent groups and the Title I District Parents' Advisory Council (DAC) to encourage information networking among parents. The DAC also advised on strategies to connect with parents through community and faith-based groups. Staff also met with other parent groups including the Title I Focus Group and the Parents' Focus on Student Achievement Group.

Rochester's District Advisory Council to Title I (DACT) has been in place for over 30 years. Recently, this group has changed its focus to include NCLB. Members attend regional and national conferences and provide early input to district leaders about other districts' interpretations and implementation efforts. District staff and leadership make themselves available to DACT members and rely on their outreach efforts. DACT sponsors parent conferences on NCLB and provides information to parents about SES opportunities. To ensure continuity of the message to all parents, DACT also includes at its monthly meetings the parent liaisons, who coordinate school-level outreach efforts.

It is critical for parents to become actively involved with their own child's education, and districts can foster widespread commitment to improving school and district achievement by creating ways for parents to engage at the decision level. Parents can then begin to take ownership over the achievement in their child's school and community. Such an effort can have a major impact on parent awareness about and participation in supplemental service programs.

A literal reading of the SES provisions might seem to suggest a fairly passive role for school districts. The state approves providers and, with provider input, identifies which areas of the state each provider will serve.⁵ The district gets this list to parents of eligible students, who select a service provider for their child. The district then contracts with the providers who have been selected. However, this picture does not match the experience of the districts in this study, each of which found it valuable to take a more proactive role.

When districts reach out to providers, they can get more up-to-date, complete, and accurate provider information to share with parents. Districts can also give providers important localized information, such as whether they will be able to work at school sites, and how or if they will be able to get information about student academic performance, both past and ongoing. Clear district-provider communication, starting at first connection, can lead to specific agreements and contracts that smooth the way for and support effective services to students. From this solid base, even stronger programs and partnerships can be built over time. Especially when the district itself is a provider, it is essential that all providers feel they have equal access to participation in the program.

First Steps

Find Out About Providers Who Will Serve Your District

Districts need to verify which providers will, in fact, serve their students. The need for districts to seek current information from providers may have been especially acute in 2002-03, the first year of SES implementation. With what was effectively a new NCLB-driven market opening up, and with the extent of demand not yet determined, it was optimistic to think that initial provider interest would necessarily translate into commitment. In its memorandum analyzing SES in 2002-03, Rochester staff articulated a common problem:

"Most of the approved SES providers were not based in Rochester and could not adequately market the program from afar. They were stuck in a catch-22 situation-trying to plan for staffing a program in Rochester, but unable to staff the program until they signed up enough students to justify the staffing. Many providers opted to drop out for a year, while planning for 03-04."

The experiences of the other districts were similar, with some providers on the state list not able or willing to offer services locally, occasionally even dropping out after parents had signed up.

Districts will need to continue to be in close communication with providers as both the supply side and demand side evolve. The market is a moving target, shifting according to the degree of demand and the success of providers in growing their capacity to meet it.⁶

Both Los Angeles and San Diego reached out to providers on the state list in order to get more complete information. They wanted to be sure that they could give parents information about providers that was comparable, dependable, and easy to understand, as required in the guidance. Although states post information about providers on their Web sites, the categories of information they include tend to vary, and individual providers may leave some categories blank. Some providers, but not all, have brochures or Web sites, and the nature of the information they offer varies. In addition, as noted above, specific answers may vary by location and change over time.

San Diego sent a written questionnaire to providers and followed up with interviews. Los Angeles created a simple, standardized one-page template that each provider was asked to fill out addressing such issues as grade levels served, staff qualifications, and demonstrated effectiveness. The results were then compiled in a brochure sent to all eligible parents, as well as being posted on the district's Web site in both Spanish and English.

Los Angeles also held several meetings for all state-approved providers who were interested in delivering services in Los Angeles, at which staff described district procedures and answered questions (see figure 6).

Work Out Provider Access to Facilities

A critical issue is where services will be provided. In many cases, a school site is an attractive location. Parents like to have after-school services housed in the same place their child attends school; it's familiar and no transportation is required. But in deciding whether or which providers should be allowed to work at the school site, a number of factors must be considered, including adequate student management and supervision, teacher property and space, and costs to the district.

Most districts have existing policies defining what types of organizations can use district facilities, under what conditions, and at what costs. Some, but not all districts in this study made school facilities available for providers under lease agreements, while the specifics of those agreements ranged significantly. For example, in one district a provider might pay only $16.80 per classroom for two hours although in another the provider would have to pay almost $300 for a four-hour minimum period.

When considering whether to offer school space to providers and when working out lease arrangements, districts will want to weigh the impact on parent's access to diverse providers. As one provider notes, "If the cost is prohibitive and would essentially eliminate hours of instructional service, then the school site is not a viable option, [which] reduces participation in the program."

Any decision to lease school space to SES providers must take into consideration the fact that some member of the school staff will need to be available to supervise them. Minimally, schools need to be assured that no unauthorized individuals will be in the building, which means someone needs to let students and-sometimes-their parents into the building and to make sure everyone is out of the building before locking up. Especially with young students, someone also needs to make sure that after their tutoring sessions they meet up with their "ride," which may mean letting them go with their parent or, in some cases, getting them onto a bus. All districts should already have in place facilities-use policies and procedures for any non-district program wishing to use district buildings, and a standard policy is to require anyone using a facility to pay for supervision. This applies to SES providers along with everyone else. The school principal is generally the person to assure that someone is assigned to this role.

Besides encouraging the use of school site space, Rochester has also built on its strong community partnerships to look for other available space that would be convenient for parents. The district has actively sought out the Rochester Housing Authority in an effort to link it with providers who may be interested in offering SES in centrally located community centers throughout the city.

Create a Fair Contract That Sets Clear Expectations

A contract between district and service provider that "leaves nothing to the imagination" provides a solid base for a smooth relationship, these districts emphasize. NCLB specifies a number of components that must be covered in such an agreement; others have been added in one or more of these districts. Each district has a detailed written agreement, ranging from the 25-page "Supplemental Services Master Contract" in Los Angeles to Toledo's eight-page agreement.

Some of the core specifics have to do with the learning goals set for each student, the timeline for achieving them, and how progress will be measured and reported to parents and the districts. These elements are generally laid out in a student learning plan, following a format set by the district, which is incorporated by reference in the contract. See subsequent section "Set Clear Goals and Monitor Progress" for further discussion and examples.

According to NCLB law and guidance (Non-regulatory Guidance, Section G-2)^*, the district contract with a service provider must include:

1. Specific achievement goals for the student, which must be developed in consultation with the student's parents [Section 1116(e)(3)(A)];
2. A description of how the student's progress will be measured and how the student's parents and teachers will be regularly informed of that progress [Section 1116(e)(3)(A) and (B)];
3. A timetable for improving the student's achievement;
4. A provision for termination of the agreement if the provider [does not] meet student progress goals and timetables [Section 1116(e)(3)(C)];
5. Provisions governing payment for the services, which may include provisions addressing missed sessions [Section 1116(e)(3)(D)];
6. A provision prohibiting the provider from disclosing to the public the identity of any student eligible for or receiving supplemental educational services without the written permission of the student's parents [Section 1116(e)(3)(E)]; and
7. An assurance that supplemental educational services will be provided consistent with applicable health, safety, and civil rights laws (see C-3 through C-5).

Once an SES provider has been placed on a state's approved-provider list, districts may not require that it meet additional criteria or go through an additional approval process before providing services in the district. Districts can, however, require that providers abide by applicable local health, safety, and civil rights laws.

Interpreting these guidelines, districts are routinely including in their contract such specifics as staff clearance requirements (e.g., fingerprinting and background checks), insurance needed, and child abuse reporting requirements.

Some district contracts also specify the district's own obligations. One states explicitly that transportation will not be provided for students. One includes a statement that a provider must serve all students who apply, up to maximum capacity.

Billing arrangements generally require the provider to submit a listing of the services that have been provided, for whom, for how many hours, and at what cost. Los Angeles has developed an attendance form and uses that form and the student's progress report as verification of services. One district has specifically stated that there is "no up-front money," specifically eliminating payment for services like student-accessed online tutorials that they found some students didn't actually use.

Districts should consider the impact of their billing policies on providers, especially those that are smaller or newly established. Often operating with less capital than larger or better-established tutoring services, these providers may not be prepared to handle large amounts of paperwork or able to float expenses during longer billing cycles.

In all instances, districts must strive to be impartial brokers in dealing with and communicating about SES providers. Among other things, this means paying close attention to the possibility of negative unintended consequences resulting from district policies (e.g., as in billing policies, noted above, that might eliminate small providers from the mix available to parents). When the district itself is also an SES provider, it is especially important that it not inadvertently set up a system giving it an unfair competitive advantage over other providers.

Going Deeper

Increase Communication and Coordination Between Providers and Schools

Because schools and teachers have regular access to parents and students, it is important for districts to link providers to schools. The key step in doing so is to encourage schools to take ownership of SES implementation. This can be as simple as having schools generate their own version of the district SES letter to parents. It can also entail something more elaborate, as in Rochester's decision to create a Title I-funded position at each of its most impacted schools for the purpose of coordinating the various academic interventions, including SES, that might be in place for individual students. A key objective is to ensure effective communication around services so they are appropriately targeted, while eliminating unnecessary redundancy, both in communication and, more importantly, services.

Districts should also work to link providers to classroom teachers. One way to do so is in the exchange of data regarding the student learning plans. Although most districts supply providers with students' most recent standardized test scores, many classroom teachers can provide a more comprehensive understanding of students' skill levels and learning needs than is available from assessment scores alone. San Diego, Forsyth County, and Rochester encourage teachers to provide student data to providers to aid in the development of individual student learning plans. In Forsyth, SES tutors were encouraged to contact teachers, who had ready access to recent achievement data through the district's TestTrax system. Several teachers reported a desire to have even more contact with tutors in the future. Providers note that this cooperation helps them set targeted goals with parents and students.

These efforts are critical in supporting the work of providers and enlisting schools in implementing SES. Initially, they allow districts and schools to build stronger relationships with providers, but they also assist with fully integrating SES into the district's improvement strategy and with monitoring providers' impact on student achievement. Most importantly, they allow for the transfer of information that helps everyone do their best in helping students learn.

Expand the Number and Type of Providers

Having a broad range of providers can increase the opportunity for parents to find the best possible fit for their children, as well as ensure that all interested parents can be served. To this end, districts can build on their existing partnerships and contacts in the community, encouraging more organizations to apply to the state as SES providers. Districts should consider approaching nontraditional providers, which may include county agencies, colleges and universities, and community- and faith-based organizations. In San Diego, a nearby university had been piloting an extended learning program in three district schools. Pleased with the partnership, the district proposed that the university apply to become an SES provider, which it did. San Diego also worked out a partnership with a local nonprofit that was already transporting district students to its own after-school program. Under the new partnership, the organization agreed to let SES providers use its facilities so eligible students in the after-school program could receive SES tutoring on-site. See figure 7 for resources to help community organizations become SES providers. (See, also, later discussions of working with community groups to get the word out.)

Some districts may find themselves with relatively few SES providers willing to work with their students, compared to the number of state-approved providers identified as serving the district's geographic region. In such instances, the district may want to explore the underlying reasons by interviewing or surveying providers. If providers identify certain district policies or practices as problematic, the district can review them and determine whether they are important to keep as is or could be reasonably modified with the aim of developing a broader pool of SES providers from which local parents can choose.

Districts with isolated rural schools may face the greatest challenge in ensuring that parents of SES-eligible children have diverse providers from which to choose. If the time required to travel to a school is long and the number of students seeking SES services is relatively small, providers may decide it is not worth their while to offer services at the school even if they were given space. Distance learning programs are often mentioned as an option in these circumstances, although such programs require an adequate technology infrastructure and the ability to assign personnel to supply on-site support for students during their SES sessions. If a district lacks these, it may want to consider partnering with the closest education services agency (ESA), known in different states as county offices of education, direct service districts, or boards of cooperative educational services. Some ESAs may already have the required technology and staff to support online learning. ESAs may also be well-positioned to become SES providers themselves, especially if they could serve students from multiple schools or from more than one district.

* The Department's of Education's Supplemental Educational Services Non-Regulatory Guidance is available as a pdf at: http://www.ed.gov/policy/elsec/guid/suppsvcsguid.pdf

Parental choice is the core of NCLB's supplemental educational services provision. The right to choose among educational supports for their child offers parents a powerful voice and directly involves them in some of the most important experiences in their child's life. Informing parents of their SES options starts that process. Ensuring that parents understand the options and can act on them is essential to fulfilling the intent of NCLB.

Parents are their child's first and most important teachers, and they have a vital interest in their child's educational progress. Some are able to hire a tutor when their child is falling behind or struggling. Prior to NCLB, low-income parents rarely had this option. Now these parents have the opportunity to select tutorial help for their child.

To do so, they need information: What services are available? Who will provide the services? How do I enroll my child? Districts are responsible for anticipating and answering these and other questions about SES. To help parents make an informed choice, they must convey the information clearly and fully. Although written communication with parents needs to meet legal requirements, the language should also be readily accessible to a range of parents with differing education and language backgrounds themselves. Any communication should present a straightforward and encouraging perspective about NCLB supplemental educational services.

First Steps

Communicate Options Clearly

• clarification of NCLB supplemental educational services,
• explanation of how students become eligible for them,
• notice that the services are free,
• where to return an accompanying application,
• the district's timeline for enrolling,
• the district's process for notifying parents about enrollment dates and start dates, and
• who to call with questions.

Figure 8. San Diego Enrollment Form

Either in the letter itself or in an accompanying publication, the district must include information about the providers that are offering services in the district and how to contact them. Because the services are free to eligible students, parents need not know the costs. But in choosing a provider, they will need to weigh cost- related information, such as how many sessions their child could have with each provider, how many hours per session, and how many other children will be taught at the same time. The district should include this kind of information in its communication to parents.

Along with its letter, San Diego sent a list of SES providers and a parent application form that offers brief descriptions of each service provider. Using the application enables a parent to enroll his or her child and select a provider in one step (see figure 8).

Los Angeles also mailed a basic informational letter home to the parents of SES-eligible students. With the letter was enclosed a provider selection brochure giving parents very clear, comparable information about each of the 26 providers that offer services in Los Angeles. The district wanted to give parents fair and parallel information about providers, whether they were a large corporation or a small community- or faith-based organization. For that reason, each provider was allocated one page in the brochure and was given specific guidelines and template instructions for describing its program: In developing its description, the provider was asked to address ten questions about its services and to include contact information. (Figure 9 identifies key provider information parents need in order to make a good SES choice for their child.) The brochure included fall and spring deadlines, a notice that the brochure was available in five languages in addition to English and Spanish, and an "official request form" that parents were encouraged to use to select a provider and to sign up an eligible child for services. A self-addressed envelope was included so parents could just drop the application in a mailbox. By making the return process easy and uncomplicated, Los Angeles hoped to persuade parents to take advantage of this new opportunity for their children. Los Angeles also sent reminder mailings designed to be eye-catching, such as the flyer in figure 10, which announces information available at the school site. The district's Web site includes, in both English and Spanish, background information on SES along with all provider descriptions from the brochure.

All of the districts in the study found that getting basic information to all parents of SES-eligible students was a challenge. Having accurate and up-to-date addresses in the district's student database is essential to avoid wasting time and postage.

Enlist Schools in a Campaign to Reach Parents

Generally speaking, parents' most trusted connections to their school district are at their child's school, with the principal or classroom teachers. As required by NCLB provisions, Toledo initially sent a letter to parents of SES-eligible students informing them of the law and encouraging them to request supplemental services for their children. But low response to this mailing caused Toledo's Title I leaders to consider that the district might have more success getting parents' attention by engaging schools in a collaborative effort to get the SES message out. Anticipating an 80 percent turnout for the November 2003 parent-teacher conferences-a number consistent with the district's past attendance patterns-Toledo's NCLB facilitator identified the conferences as a key opportunity to reach parents of SES-eligible students. She met with principals and teachers during regular staff meetings to lay out the plan: The district's NCLB staff would prepare classroom packets that included an individually addressed letter and a provider-description brochure for parents of SES-eligible students. Classroom teachers would hand these packets to parents during the conferences, explain the opportunity, and encourage them to request services for their children. Successful implementation of that plan proved a turning point for Toledo's SES participation.

Toledo's Title I leaders also kept schools informed about the numbers of their students enrolled in SES and urged teachers to contact parents of students not yet enrolled. The focused and consistent message to school staffs began to persuade them of the importance of SES in their overall school improvement efforts. Even principals and teachers who might initially have been skeptical about the value of SES came to embrace it as an essential opportunity for all of their eligible students. That enthusiasm revealed itself as they communicated with parents about taking advantage of the opportunity. Involving the principals and teachers at targeted schools in this team effort has resulted in many more children receiving the extra academic support that is rightfully theirs (see figure 11).

In Los Angeles, a similar campaign was initiated after an extensive mailing to parents yielded disheartening returns. Acknowledging that mail from "the district" might not grab parents' attention, district leaders held strategy meetings with principals and Title I coordinators from the 104 identified "program improvement" schools. Together, they undertook an information campaign to get parents' attention and encourage them to return the request form in the booklet they had received. In addition, staff prepared SES supply boxes for each school. The boxes contained extra provider selection booklets in English, Spanish, and five other languages; mailing labels for all eligible students; a CD listing all of the eligible students; and informational flyers in multiple languages to be sent home with eligible students. Schools became local information centers and principals and teachers communicated with parents about SES opportunities by holding parent meetings, greeting parents when they brought their children to school or picked them up, and by calling eligible parents to urge them to enroll their children in supplemental services.

Going Deeper

Expand Communication Channels

The districts in this study have discovered through many attempts that merely notifying parents by letter, as called for in NCLB guidelines, is insufficient. Because of the low return from their initial mail campaigns, every district in the study started seeking other methods and media to let parents know about SES opportunities. As noted above, connecting with parents through their children's schools is a practical strategy for informing them of their options. The broader school community offers other possibilities to connect with parents and let them know about their SES options. Parents can communicate the information through their informal networks-for example, as one mother tells another and she, in turn, tells yet another, the news spreads throughout the community. Natural gathering places like community centers, churches, health centers, and commercial shopping centers also lend themselves to serving as information and recruiting locations.

Other marketing strategies that districts have used include:

• articles and ads in local newspapers, especially in neighborhood and ethnic publications;
• press releases and news conferences;
• public service announcements on radio and television;
• flyers and posters (see figure 10, for example);
• interviews on radio and television;
• refrigerator magnets; and
• postcard reminders.

When effectively deployed, these strategies successfully accomplish six important communication goals:

1. get parents' attention;
2. inform them about their SES options;
3. help them understand how to access their options;
4. motivate parents to take action to exercise their options;
5. encourage them to follow and communicate about their children's progress; and
6. influence them to provide evaluative feedback regarding the impact and quality of the services their children receive.

Increase Community Involvement in Getting the Word Out

Using the natural informal communication networks of a community may be one of the most effective ways of getting information to parents about the NCLB supplemental educational services available to their children. With many people, word-of-mouth recommendations from credible people seem to be an effective way to sell an idea or a service.

For example, with the help of its very active Title I District Advisory Council, Rochester began to get the word out about SES opportunities through its existing parent information networks. Parents whose children were receiving NCLB supplemental educational services were asked to "spread the word" to other parents about the opportunities available. Parent liaisons at SES-eligible schools began calling parents and encouraging them to sign their children up for SES. The parents' marketing strategy worked: Many more eligible students began working with tutors and other providers.

The Rochester parents also suggested taking advantage of the district's popular regional parent outreach centers by using them as information and enrollment centers for SES. Having information about providers and enrollment forms available in these parent-friendly environments was another successful strategy for attracting and informing parents about the services their children were eligible to receive. Similarly, Forsyth's Transition Center has been successful in overcoming cultural and language barriers and has had similar results recruiting parents to select providers and enroll their children in supplemental services.

Working with respected community leaders and community- and faith-based organizations is another approach that several districts have used to encourage parents to take advantage of their new options. Community leaders and local grassroots organizations have well-established avenues into their communities and can generate a sense of urgency and importance about SES. Stressing the importance of parental choice as a newly available right for parents, community leaders can also emphasize parents' responsibility to use their opportunity to assist their children's academic progress.

Districts can also partner with any of a number of other organizations focused on improving education opportunities for children, in part, by getting relevant information to parents. To find out about specific organizations of this ilk, including each state's Parent Information Resource Center, the Black Alliance for Educational Options, the Hispanic Council for Reform and Educational Options, and the National Coalition for Parent Involvement in Education, see appendix C: Resources.

Where the rubber meets the road is in providing academic instruction that boosts student achievement. SES service providers have met state requirements that include evidence of effectiveness. But how does that translate into instruction tailored to each student's specific needs that will complement and extend classroom instruction and contribute to reaching standards on state assessments?

NCLB regulations give states the responsibility of ensuring that SES services are high quality and effective in raising students' academic achievement. Each state must identify methods of measuring each provider's impact on students' academic learning, and it may enlist the assistance of local districts to collect data and supply input. Many states are just now gearing up to meet this requirement, which will inform decisions about which providers remain on the list. Although states must take the initiative to evaluate providers, they may solicit districts to collect student participation data, feedback from parents, and other information from district contracts with providers. According to the U.S. Department of Education's non-regulatory guidance on SES, "The State may want to request assistance from its LEAs or may want to handle this monitoring [of continued effectiveness by the provider] at the State level."⁷

Although each state has chief responsibility for ensuring that its SES-eligible students overall receive high quality, effective SES services, the law has embedded certain mechanisms at the local level that are also intended to help ensure the success of SES services for individual students. Most important are its requirements for individual student learning plans, for services to be aligned with state standards and consistent with district curriculum and instruction, and for progress reports. It falls to the district to make sure these elements are in place.

First Steps

Use Detailed and Specific Student Learning Plan Forms

According to NCLB's SES provisions, a district's contract with a provider must require detailed achievement goals for the individual student, a timeline for meeting the goals, a method to measure student improvement, and a schedule for informing parents and teachers about student progress. The provider must develop student goals in concert with parents. To streamline the communication and contracting process, many districts have created a learning plan template as an independent, yet integral, part of their contract with providers; this ensures that learning plans, although tailored to the needs of each student, are also consistent across providers and students in what they address. Districts have discretion over how these plans are formatted and what, if any, additional content will be included, resulting in a range of templates and a variety of data being collected across districts. But by and large, the templates ensure two things: that unique goals are established for each student and that providers indicate the evaluation tools they will use to document student improvement.

In Los Angeles, the student learning plan calls for providers to work with parents to identify achievement goals, services a student will receive based on those goals, and the measurement and assessment tools to be used to track progress. Such tools may include a pre- and post-test, a daily written or oral diagnostic, or other assessment measures determined or developed by the provider. See figure 12 for an example of a student learning plan that has been completed. The goals are generally drawn from the provider's understanding of the state standards that are applicable to each student's needs.

Rochester has created a slightly different Supplemental Education Plan template for providers to use in developing a learning plan for each client. Providers do not simply select a set of mathematics or reading goals from a list of state or district standards. Instead, they select an area of service (e.g., English language arts, mathematics, IEP, 504, or LEP) and then, in consultation with parents, determine the unique skills to target for each student. Providers also indicate how student progress on these skills will be measured and how often parents will receive reports on student achievement as required in the SES provisions.

Share Data and Set Student Goals Aligned with the District's Instructional Program

An important step in developing student learning plans is ensuring that, as required in the law, SES services are consistent with the district's curriculum and instruction and are aligned with the state's academic standards. In approving providers, states must first ensure that a pro-vider's services are "consistent with the instructional program of the LEA [local education agency] and with State academic content and achievement standards."⁸ This does not mean, however, that there is only one acceptable service approach. Far from it: SES tutoring might amplify and reinforce the student's classroom instruction, giving more practice and perhaps a different explanation or perspective; or, a student might have a weakness in some very specific skill areas and need concentrated practice in those areas. Whichever strategy is pursued, it should be deliberate and informed by a thorough understanding of the standards the student needs to master and by as much diagnostic information as possible, with input from parents and classroom teachers.

To facilitate development of the student learning plan and help ensure that it addresses the student's individual needs, districts may need to seek parent permission to share student data with the SES provider. In addition to sharing assessment data, they may also try to link the provider to the classroom teacher so that the classroom teacher can share additional information-again, with parent permission. Close coordination at this level also helps ensure that SES complements, rather than duplicates or conflicts with, district efforts. Once students are receiving services, districts can encourage continued communication between teachers and SES tutors about student progress. Such efforts are critical in supporting the work both of providers and classroom teachers. Not incidentally, they also help build greater school-level commitment to the SES program.

In Forsyth, data-driven decision-making is a key component of the district's long-term strategic planning, of schools' day-to-day operations, and of teachers' instructional planning. So, not surprisingly, Forsyth has encouraged its SES provider to contact students' classroom teachers and solicit additional achievement data beyond what the district supplied. The provider developed a Teacher-Tutor Data Collection Sheet (see figure 13) that was used to determine students' initial skill set and to provide feedback to teachers and the district about student progress. After obtaining parent permission, provider staff would contact students' regular classroom teachers and request input on student performance in reading, writing, and mathematics. Tutors also had an opportunity to provide their own input on the same indicators after meeting with and assessing students. This sheet also included the results of the diagnostic test conducted by the provider, who used the data to develop student learning plans and report to the district on students' progress in the program. Teachers also had access to this data to inform their work with students. By creating a feedback loop between teachers and tutors, Forsyth is able to ensure instructional goals are being met.

In several of the districts studied, the state department of education required them to maintain a learning plan for all students at risk of not meeting state grade-level standards. In addition, the districts' own strategic goals called for a learning plan for such students. To avoid piling one learning plan on top of another, the districts are trying to devise their own systems to coordinate the information in all of the learning plans.

San Diego and Rochester are ahead of the curve here because they have chosen to use previously required student progress plans as a base for the SES plan. Under Rochester's Academic Intervention Services (AIS) system, mentioned earlier, instructors are required to monitor and document student progress on state learning standards in an AIS plan. Because the indicators on the AIS progress report are identical to those that need to be measured in a Supplemental Education Plan, the AIS report now serves double duty as the basis of the district's Supplemental Education Plan. In San Diego, the Learning Contract is a key component of the district's Blueprint for Student Success. A Learning Contract must be developed for every student at risk of not meeting grade-level standards in reading or math. Because SES is one of many interventions supporting students at academic risk, San Diego has made students' SES plan a component of their learning contract rather than a separate stand-alone document.

San Diego and Rochester, along with Los Angeles, are also in the process of modifying their student achievement data systems to coordinate each student's learning plan information into a coherent record that will be a meaningful and useful guide to academic intervention.

Day-to-day tracking efforts, like reviewing student attendance, can enable districts to get an early handle on the benefit a student receives from SES. A student's attendance record suggests whether he or she is responding to an SES program. Using attendance as a metric encourages providers to maintain communication and positive relationships with parents, ensuring students' consistent participation. It can give an early warning of the need for extra attention.

Los Angeles is one of several districts that track student attendance as a measure of SES program effectiveness. Rather than using separate forms to collect the data, Los Angeles asks that providers include the information as part of the invoice they submit for payment.

Forsyth has also found an efficient way to monitor student attendance in SES: SES tutors report attendance on the Teacher-Tutor Data Collection Sheet. Making attendance reporting a natural part of the work providers do with students-as opposed to requiring a separate report-can make the process easier for both the district and SES providers.

Set up and Use Regular Progress Reports to Parents and Teachers

Providers must be accountable to parents by giving them regular reports on student progress. As required by NCLB, classroom teachers must also receive progress reports to help ensure that "students are improving their academic achievement and that instructional goals are being met."⁹ Each district in this group has progress reporting in place, with slightly different features and timelines.

To facilitate the billing and reporting components of its provider contract, Los Angeles requires all providers to send progress reports to a student's parents and teachers after every 15 hours of service. Providers can report more frequently if they choose to or if parents request it. But this length of time gives providers a chance to work with students to meet initial goals and gives parents a chance to revise learning goals after talking with teachers or reflecting on student progress.

Forsyth's contract calls for providers to supply parents and teachers with the results of pre- and post-testing, as well as biweekly progress reports (see figure 14). In addition, the district's provider — a one-on-one, in-home tutoring service — boasts that its tutors meet with parents at the end of each tutoring session to discuss the work they did and homework expectations for students. This provider encourages parents to monitor students' work and even to replicate some of the teaching behaviors used by tutors. All of this is intended to make sure students receive the greatest benefit from the SES program.

Going Deeper

Evaluate Student Progress on District Assessments

Many districts are in the midst of strengthening their assessment systems and database capacity so that data on student performance and other measures can be better used to guide decisions. Benchmark assessments throughout the school year, perhaps paced by units in the curriculum, are increasingly common. These assessment tools can be used to judge the contribution of different instructional programs, including SES.

In 2003, Rochester was in conversation with Tungsten Learning-a division of Edison Schools-and Scantron to contract to use their systems, which provide benchmark assessments aligned with state standards. The Tungsten system assesses students monthly, while Scantron would do so every nine weeks. District leadership approved a pilot of both programs at various grade levels in a limited number of schools. Rochester intends to use one of these systems to measure provider impact on student achievement over the next few years. Most providers are already using their own pre- and post-test assessments to evaluate student progress; however, lacking early evaluation data from the state, the district is trying to establish a way to supply parents with objective information that can be compared across providers.

Figure 15. Rochester Parent Survey

SURVEY PARENTS

Parents are a key source of information to evaluate SES. Some districts have begun surveying parents about their experiences with SES providers. Like student attendance, the degree of parent satisfaction with a provider is an important early indicator of effectiveness. Los Angeles and Toledo use parent comments and evaluations as a means of determining a provider's effectiveness. Parents' satisfaction with the progress their children are making as a result of the services they are receiving is a fundamental indicator of how well providers are serving students' needs.

The staff overseeing Rochester's SES implementation are committed to finding meaningful ways of evaluating both SES providers and district SES implementation. The district conducted a survey of parents whose children received supplemental services during 2002-03. This survey asked parents to rate both the provider and the district's NCLB Office (see figure 15). Staff have discussed reporting the results as part of the letter or brochure to parents of eligible students. At the very least, the district will use parent feedback in reviewing its SES policies and compare results over the duration of SES implementation as part of its evaluation efforts.

Reflect on Implementation and Adjust Efforts Based on Formative Feedback

Evaluating SES is not just about student performance. It's also about districts themselves learning how best to implement the program. Many of the previous sections of this guide highlighted practices that emerged when districts were trying to improve upon initial practice. Ongoing reflection informed by a range of indicators is the hallmark of a learning organization.

In Toledo, for example, when the early mailing to parents failed to generate much response, staff considered this feedback and quickly changed tack. They switched from a largely central office outreach effort to site-based outreach, recruiting parents and teachers at participating schools to make phone calls to other parents of SES-eligible students. The district also conducted outreach during parent-teacher conferences, parent meetings, and other school-based events. This kind of responsiveness resulted in both a higher yield of eligible student participants and a school community more actively engaged around implementing SES.

State departments of education have been given an essential role in establishing effective supplemental educational services under No Child Left Behind. Chief among their responsibilities are soliciting, screening, and approving a broad array of SES providers and implementing a system for then monitoring the quality and effectiveness of the services offered by those providers. In addition to these direct responsibilities, state departments have an implied duty to support districts striving to meet both the letter and the spirit of SES provisions.

Meet District Data Needs

States need to provide districts accurate and timely data on school eligibility and on which providers have met state requirements to supply SES services to students.

Provide School and Student Data

Although schools can anticipate certain aspects of their improvement status (e.g., once they have missed AYP two years running and have had to offer school choice, they may need to offer SES the following year), they rely on states for ongoing information about their improvement status. States should release provisional improvement lists as soon as possible following administration of the state assessment, minimally before the beginning of the next school year. They should also expedite their appeals process.

Supply Approved Provider Lists

To grow their provider list, states may want to maintain a rolling provider-application deadline, while keeping in mind districts' need to inform parents of their SES choices in a timely fashion. States are required to give school districts updated lists of approved providers in their "general geographic locations." To be as helpful as possible to districts whose job it is to inform parents about their SES choices, states should also consider soliciting and sharing additional provider information. In addition to more specificity about where providers are willing or able to work (e.g., statewide, only in certain regions or cities), state agencies may also share more detail about each provider's evidence of effectiveness and type of service offered (e.g., one-on-one tutoring, small group instruction, any content specialties, grade level served, any set locations of service, ability to serve special populations). Most states maintain a Web site with up-to-date provider information.

States should also consider ways to attract a diverse pool of providers to serve their SES-eligible students. By reaching out to entities across the state, including faith- and community-based organizations, charter and private schools, universities, and others, states can better ensure a large and diverse pool of providers. In the same vein, states may want to consider holding technical assistance workshops to help potential providers apply.

Monitor Districts' SES Implementation

States should consider how they will monitor district implementation of SES. At a minimum, they should include SES as part of their overall Title I monitoring plan. Additionally, to support effective SES programs, states might consider offering their LEAs a model provider contract and a model letter to parents. States should also be prepared to address any SES implementation concerns brought to their attention by parents, community groups, providers, or others with interest in ensuring successful SES programs.

Move Forward on Provider Evaluation Efforts

Under NCLB, state educational agencies are responsible for monitoring the quality and effectiveness of SES providers and should move quickly to initiate the process if they have not already done so. In developing their evaluation systems and indicators, states may want to survey local districts, many of which have already begun collecting data on provider effects by such means as monitoring individual student learning plans, surveying parents, and administering uniform benchmark assessments. To strengthen their provider monitoring and evaluation efforts, states may also want to enlist one of their universities as a partner or collaborate with neighboring states. One state requires all providers to assess their students using a pre-test and a post-test. Although providers can choose their own assessment tool, they must document how the assessment is aligned to the state standards. This same state has developed and is about to implement a statewide, Web-based SES student tracking and reporting system through which providers can record student enrollment, attendance, and progress. As incentive for providers to use the system, it also includes an invoicing function for providers. Once the system is integrated into the state's overall student data system, the SEA will be able to look at student progress in supplemental services in the context of other information about students' overall school experience and achievement. In conjunction with this electronic system, however, the state intends to continue face-to-face provider reviews to make sure services reflect what the SES provider has committed to providing.

Outreach to Parents

States may want to begin outreach campaigns tolet parents know about SES and their child's eligibility. State-level outreach might include conferences, public service announcements, printed materials, and other communication.

Collaborate With Other States

As noted above, states can profit from working with one another or informing each other on any number of SES-related issues, such as discussing how to refine their provider screening processes or identifying longitudinal measures of student progress under SES. Inter-state sharing of information could be an important first step in finding efficient solutions that build on others' lessons learned.

Encourage District Collaboration

Similarly, states should encourage cross-district collaboration and sharing of SES experiences and strategies. Neighboring districts or districts with similar demographics often face common challenges (e.g., rural districts may face similar transportation issues or have difficulty attracting any service providers) and could benefit from hearing each other's strategies or, in some instances, from approaching common problems collaboratively.

The development of this guide was initiated and directed by Nina S. Rees, Deputy Under Secretary of the Office of Innovation and Improvement at the U.S. Department of Education. Sharon Horn was the project manager.

An external advisory panel provided feedback to refine the study scope and prioritize issues to investigate, as well as reviewed and provided feedback on drafts of the guide. The panel consisted of Mike Casserly, Executive Director, Council of Great City Schools; Jeffrey Cohen, President, Sylvan Education Solutions; Steve Fleischman, Principal Research Scientist, American Institutes for Research; Jane Hannaway, Director, Education Policy Center, The Urban Institute; Laura Hensinger, Colorado Department of Education; Frederick M. Hess, Resident Scholar, American Enterprise Institute; and Lana Watkins, NCLB Coordinator, Philadelphia Office, Black Alliance for Educational Options. Drafts of the guide were also reviewed by district staff of the five districts included in the underlying study and by a sample of SES providers working in those districts.

Staff in the Department who reviewed drafts include: Tom Corwin, Michael Petrilli, Cynthia Dorfman, Stacy Kotzin, Meredith Miller, Christie Wolfe, Phil Rosenfelt, Susan Wilhelm, Fran Walter, John Gibbons, Jeff Sims, and Margaret West Guenther.

This guide was written, designed, and based on a report by WestEd.

WestEd is a nonprofit research, development, and service agency committed to improving learning at all stages of life, both in school and out. WestEd has offices across the United States and also serves as one of the nation's ten regional educational laboratories.

WestEd's partner in developing this series of research reports and innovation guides is Edvance. Created by the American Productivity and Quality Center, Edvance is a resource for process and performance improvement with a focus on benchmarking, knowledge management, performance measurement, and quality improve ment initiatives in education.

The five districts cooperating in the development of this guide and the report from which it is drawn were generous with both their time and attention to this project. We would like to thank the district superintendents and the many district staff members who were instrumental in coordinating and participating in the site visits that inform the report and this guide.

Forsyth County Schools
1120 Dahlonega Highway
Cumming, GA 30040
http://www.forsyth.k12.ga.us/
Paula Gault, Superintendent
Melinda Ness, Special Programs Coordinator

Los Angeles Unified School District
333 South Beaudry Avenue, 24th Floor
Los Angeles, CA 90017
http://www.lausd.k12.ca.us/
Roy Romer, Superintendent
John Leichty, Associate Superintendent
Becki Robinson, Program Specialist, Extended Learning

Rochester City School District
131 West Broad Street
Rochester, NY 14614
http://www.rcsdk12.org/
Manuel Rivera, Superintendent
Terry Hofer, Director, Grant and Program Accountability

San Diego City Schools
Eugene Brucker Education Center
4100 Normal Street
San Diego, CA 92103
http://www.sdcs.k12.ca.us/
Alan Bersin, Superintendent
Christiane McPhee, Extended Learning Programs

Toledo Public Schools
420 East Manhattan Boulevard
Toledo, OH 43608
http://www.tps.org/
Eugene Sanders, Superintendent
Lu Lane, Compensatory Programs Director
Cindy Kline, NCLB Facilitator

WestEd
730 Harrison Street
San Francisco, CA 94107
http://www.wested.org/
Glen Harvey, Chief Executive Officer
Nikola Filby, Associate Director, Regional Laboratory Program

Edvance
123 Post Oak Lane, Floor 3
Houston, TX 77024
http://www.edvance.org/
C. Jackson Grayson Jr., Chief Executive Officer
Kristin Arnold, Project Director

To say that this Georgia district has been growing is putting it mildly. In 1991, Forsyth County Schools' enrollment stood at 8,000. Today, a dozen years later, the district serves over 22,000 in 23 schools, and its student population is expected to grow to 34,000 within the next five years. District leaders have been aggressive in developing the infrastructure necessary to respond to Forsyth's dramatic growth. One important strategy was a decision in 1996 to join the Center for Leadership and School Reform's Standard Bearer Network, under the direction of noted reformer Philip Schlechty. Through its participation in this 10-district pilot effort, Forsyth has had access to collaboration and resources otherwise more difficult to find. With the help of this network, the district has been increasing its use of data-driven decision-making and reliance on research to guide strategic planning and practice.

Research about how best to support students who have fallen behind academically has informed Forsyth's implementation of extended learning opportunities (ELOs). Its Early Intervention Program (EIP) provides supplemental instruction during the school day to K-5 students (irrespective of Title I status) who have below standard on the state's competency test. Its Extended-Day Program provides after-school instruction for students performing below grade level on state reading and mathematics assessments, with priority given to EIP and Title I students. All involved-district leaders and staff, teachers, and parents-view these programs as essential to the success of participating students. It was this broad recognition of the value of additional instruction and time on task that lead Forsyth to see SES as a benefit. And because district leaders and staff saw SES as a benefit, they set a positive tone for implementation.

Forsyth initially had two schools whose improvement status triggered SES. Early on, Forsyth realized that its success in getting eligible students at these schools connected to SES was contingent upon parent trust. Although the district sent letters notifying eligible parents about SES, it also recognized that, for many parents, their most important connections to the education system are their relationships at the school level, with classroom teachers and principals. Thus, Forsyth has encouraged and supported school-level outreach. The district also reaches out to new parents through its recently established Transition Center, which provides assistance on such things as student registration, placement testing, and, not least, signing up for SES. Because many new district families are Hispanic, the district sought out as Center coordinator a particular bilingual teacher-on-assignment who has extensive credibility in the community. Parents can thus receive information from someone who speaks their native language and who is well known and well respected. This targeted outreach has proved effective: 99 percent of SES students in 2002-03 were classified as "English as a Second Language" students.

The district was approached by four service providers but two dropped out early on: one determined that the distict's per-child allocation was insufficient to cover its costs and the other, a community-based organization, concluded that it did not have adequate internal resources to carry out the services. The two that remained were different franchises of the same parent company. When that parent company declared that Forsyth County was in the service territory of only one franchisee, the district was left with that one provider. Although district staff have consistently made themselves available to organizations considering becoming SES providers in Forsyth County, to date no other providers have signed on. Although the district had no say in the provider selection, it saw the potential of partnering with its one provider to assist the district's neediest students, and it looked for ways to collaborate to gain even greater pay-off. One strategy for leveraging the SES was to facilitate the ongoing exchange of data between a student's tutor and classroom teacher.

Forsyth's SES tutors use lesson plans available on the Georgia Department of Education Web site to ensure instruction is aligned both to state standards and to classroom learning. Additionally, instructors supplement their lessons by assisting students with homework assigned by classroom teachers. In this way, the program targets students' distinct abilities and needs. The program also uses an in-depth assessment to measure starting points and monitor progress. The results of this diagnostic, along with biweekly progress reports, are provided to parents and teachers.

Forsyth has instituted a continuous feedback loop to support the efforts of both classroom teacher and tutor. Teachers are encouraged to work with tutors and parents to develop the required student learning plans and to review the results of the provider's pre- and post-test. They are also expected to supply tutors with current assessment data: In 2003, the district launched TestTrax, which enables parents, teachers, and school leaders to access standardized and standards-based test data in digestible reports to better understand how students are performing. Classroom teachers provide that data to SES tutors.

The Los Angeles Unified School District (LAUSD) is a large, sprawling school system covering some 700 square miles and serving nearly 750,000 K-12 students in 713 schools. In addition to the city of Los Angeles, the district encompasses 25 other cities and unincorporated areas in Los Angeles County. Many district schools and the students and families they serve are affected by the challenges found in most large urban areas, including severe poverty, crime, and violence. Compounding these challenges for the district are overcrowding, facilities in need of repair, and staffing difficulties. Many students enrolled in these schools also contend with issues related to immigration and language. The predominant ethnic group in the student population is Hispanic, representing over 72 percent of the enrollment, with African American students making up 12 percent, white students 9 percent, and Asian American students 6 percent. Taken together, Hispanic and African American students perform at significantly lower levels than all other groups-resulting in a major achievement gap that the district is aggressively tackling.

Although some LAUSD schools are showing steady progress (13 Title I schools earned a Title I Achieving Schools Award for having met and sustained their Annual-Yearly-Progress targets for three consecutive years), 106 schools are currently designated as "Program Improvement Schools." SES is an important avenue to help serve the children in these schools.

Given the size of the district and the number of students eligible for SES-164,000 the first year-implementation of the new program has been a tremendous undertaking, involving virtually every district department. The district placed SES within its Beyond the Bell (BTB) department, which managed its existing extended learning program. But to ensure that this important new program did not get lost in the mix, LAUSD appointed a full-time SES manager and an assistant. They oversee all of the information and recordkeeping systems and monitor the communication loop among the students, their parents, the students' classroom teachers, SES providers, and BTB. Managing information has been a major focus: To effectively carry out the massive process of informing parents, the district modified both its information management systems and its student database to include specific fields for SES data. It has also created additional databases to monitor and verify student registrations, parental provider selections, SES facilities information, and provider recordkeeping.

Twenty-six SES providers offer academic assistance to Los Angeles's students, including LAUSD's own Beyond the Bell Learning Centers. Given the extremely large number of eligible students, the district decided to create a "master" contract for each provider, rather than a separate one for every student served; in addition to signing this umbrella contract, providers must complete a learning plan for each student. Providers are also required to document student attendance and to send progress reports to a student's parents and teacher after every 15 hours of service. To notify eligible parents, LAUSD mailed an explanatory letter, along with a provider-selection booklet and an "Official Request Form" in English and Spanish. Upon request, the information was also sent out in one of five other languages. Following a disappointingly low response to that initial mailing, BTB leaders held strategy meetings with principals and Title I coordinators from the program improvement schools and planned a more intensive marketing effort centered on school-to-parent communication. Each school was given an SES supply box containing, among other things, a CD listing all eligible students, mailing labels for each one, extra provider-selection booklets (available in seven languages), and informational flyers in multiple languages to be sent home. Urging parents to take advantage of SES's free tutoring, the provider brochure and one of the district's flyers caution, "Don't throw away your child's future!" Principals and teachers also communicated with eligible parents through meetings, by greeting parents when they brought children to school or picked them up, and by telephoning them.

BTB staff met with a variety of parent groups, including the Title I District Advisory Council (DAC), to encourage participants to spread the word. The DAC, in turn, proposed strategies to connect with parents through community- and faith-based groups. The district also advertised SES in local newspapers, including ethnic language papers, and on radio and television. The superintendent and other district leaders held press conferences, and the Office of Communications published several news releases.

Although actual SES impact on student progress will be determined by evaluating a student's classroom performance and assessment results, Los Angeles keeps track of two other near-term indicators of effectiveness: student attendance and the usefulness and quality of information providers supply to parents and teachers about student progress after every 15 hours of service.

Rochester City School District and many of its families face difficult conditions. According to 2000 census data, out of America's 245 largest districts, Rochester had the 11th highest child poverty rate, with 38 percent of its children ages 0-17 living below the poverty line.¹ Of the nearly 37,000 students enrolled during 2002-03, 78 percent qualified for the free lunch program and 80 percent for Title I services.² All of Rochester's 57 schools are eligible for Title I services, and a third of its students, at 8 schools, qualify for SES.³ The poor achievement in 21 schools that have not met AYP goals has caused the district itself to be designated as "in need of improvement." In this challenging environment, Rochester developed SES as a means of augmenting existing district efforts to raise the achievement of its lowest-performing students.

The district's concerted outreach efforts have begun to pay off: Between the first and second year of SES implementation, the number of students requesting services more than quadrupled, rising from 429 to 1878. The district has profited from an established culture in which parents are treated as collaborative partners. For more than 30 years, local parents interested in ensuring high-quality Title I education have worked through the District Advisory Council to Title I (DACT). Although all Title I programs are required to fund a parent advisory board, the district's proven receptiveness to working with the DACT meant that when NCLB passed, the group stepped up immediately to help move it forward.

DACT members attended state and national conferences about Title I and NCLB and became knowledgeable enough to assume a critical role in providing early information to both parents and district staff about NCLB's various provisions, including SES. In addition to its members reaching out personally to other parents, the council has held town hall meetings on NCLB, hosted an application night to link parents to SES providers, and written about SES in its newsletter. The DACT also coordinates its message with the parent liaisons at Title I schools. In addition to helping parents understand how to support their child's learning, these liaisons have become an important link to parents not yet aware of SES.

To ensure that the SES program receives the attention it warrants and the set-up expertise necessary for an important new program, Rochester has placed it in a district office that is experienced in incubating new initiatives. Once it has been solidly established, it may be moved into the office that oversees Rochester's state-mandated Academic Intervention Services (AIS), which serve students who have not passed, or are considered at risk of not passing, New York's standards-based assessments in key academic areas. Identification of a student's needs and appropriate services are made jointly by the student's parent(s) and classroom teacher and documented in an AIS plan, just as the learning needs and targeted services for students receiving SES are documented in a Supplemental Education Plan (SEP).

In Rochester's first year of SES, 11 providers were serving district students. The district is intent on getting those who work with students on the same page. To that end: The SEP template specifies the NCLB requirement that all supplemental services align with state academic standards and district curriculum. Parents are encouraged to share with the provider information from their child's AIS reports, which can be used to target instruction. The district has also created a new Title I-funded position of AIS specialist for each of its most impacted schools to coordinate academic interventions, including facilitating ongoing exchange of information between providers (both AIS and SES) and classroom teachers.

The district has also been focused on how to assess student learning under SES. Qualitatively, it solicits parent satisfaction data, and to get the big picture, it looks at students' scores on the statewide test. But the district has begun investigating the use of standards-based benchmarking assessments common to all providers. Although the district contracted for and has begun piloting such assessment in a limited number of schools, the question of how to fund them on a larger scale remains unanswered. Either way, Rochester's intent to both qualitatively and quantitatively evaluate the impact of SES reflects its commitment to keep everyone focused on the primary goal of better meeting students' learning needs.

¹2000 census data, as presented by Children's Defense Fund. Available at: http://www.childrensdefense.org/data/census00/pov/city.txt.

²District Web site: http://www.rcsdk12.org. Washington, DC: U.S. Department of Education, 2nd Edition 1999.

³Democrat and Chronicle.com (January 22, 2004). Create real reform [editorial]. Rochester, NY: Author.

In San Diego, poverty is not readily apparent to casual visitors who tend to be more familiar with the area's beautiful beaches, its world-famous zoo, and other tourist attractions. But in school year 2002-03, 56.4 of San Diego's almost 140,000 public school students were eligible for free or reduced meals, 31 of its 185 schools had such high percentages of eligible students that under federal regulations the schools rated as 100 percent eligible, and 126 schools received Title I funds. In this 200-square-mile district just north of the U.S.-Mexico border, 29.4 percent of the students are designated English learners, and more than 64 learning other than English are spoken as the primary language in students' homes.

San Diego has been working hard to close a challenging achievement gap, work that has started to yield results. District achievement data over the last six years reveal consistent improvement in student performance. For school year 2002-03, California's student achievement accountability barometer-Academic Performance Index Blueprint, which requires classroom teachers in program (API) results-showed growth in student achievement across all district schools and all student subgroups. Moreover, Hispanic, African American, and socioeconomically disadvantaged students made greater gains districtwide than any other group. But more progress is needed. Of the 126 San Diego schools receiving Title I funding in 2003, 37 had been designated "program improvement" schools by the state, and 24 of those had been in program improvement status long enough to require that they offer supplemental educational services. In the district's first year offering SES, four approved providers were available to San Diego students, and parents signed on with two of them.

District leaders attribute much of the generally positive API results to the district's Blueprint for Student Success in a Standards-Based System, a comprehensive and ambitious long-range reform plan adopted in 2000 and aimed at improving academic performance. Quick implementation of many of the Blueprint strategies meant that when No Child Left Behind (NCLB) was adopted a year later, many of the legislation's key requirements were already in place, including extended learning opportunities (ELOs). Recognizing SES as an important addition to its existing ELOs, San Diego placed SES in the Office of Instructional Support, which also oversees the ELO Department, and gave it direct, day-to-day management of the new program to the ELO director.

When it came to developing SES's required student learning plan, San Diego was ahead of the game thanks to the Blueprint, which requires classroom teachers in program improvement schools to complete a learning contract for every student at risk of not meeting grade-level reading or math standards. Regularly updated with performance data, a student's learning contract is used to guide both classroom teaching and the district's after-school academic intervention efforts. Instead of creating a separate document for the SES student learning plan, the district has chosen to incorporate the required information (e.g., SES learning goals, timeline, progress reports) into students' existing learning contract. This one-student, one-plan approach is the district's approach to keep student achievement information coherent and organized in one document that follows the student electronically.

San Diego uses several strategies to inform parents about SES. Every identified parent receives a letter, sent out in five languages, telling them that free tutoring is available and including a list of providers and an application form. When the initial mailing yielded a low response, the district got the principal and teachers at each program-improvement school to communicate directly with their parents. The district sends principals SES timelines and a list of eligible students along with SES enrollment information and application forms to keep at the school. In addition, the district's Web site (http://www.sandi.net/) offers a variety of NCLB-related information for district staff, school-site staff, and parents alike.

Rather than wait for the state department of education to provide evaluative information about providers, San Diego is developing an assessment of provider effectiveness. Thus far the district has been using two primary strategies: close monitoring of students' learning contracts to track progress, with follow-up meetings with teachers and parents regarding students' academic progress, and solicitation of parent feedback about how they think their children are doing.

Toledo Public Schools serves a diverse 84-square mile community on the western shore of Lake Erie. Among its broadly recognized assets are a world-class art museum, a major symphony orchestra, and an expanding and progressive zoo. But like many urban centers, the city has suffered from an economic decline that has continued since the recession of the eighties. Many of its residents struggle from job loss or low wages and the resulting poverty. Of the nearly 36,000^* K-12 served by the district's 62 schools, 54 percent qualify for free or reduced lunch. So it's not surprising to hear that local schools demonstrate a clear achievement gap, with the district's lowest-performing students clustered in schools serving the largest concentrations of high-poverty students.

Determined to do better by its students, during the 2000-01 school year the district embarked on an ambitious accountability agenda. When No Child Left Behind became law, district leaders quickly recognized the convergence of its goals and their own reform efforts. They saw its SES requirement as particularly supportive of their improvement goals. But not everyone so sure. Toledo's Title I schools initially saw the associated set-asides as money being "taken away from the schools that needed it most." In time, however, says one administrator, even the skeptics began to see SES "as a great way to give extra support to the kids who needed the most help... [as something] that will help our program-improvement schools meet their AYP targets." During Toledo's first year of SES, 11 schools had SES-eligible students, who were served by three of 18 approved providers.

Toledo placed its SES program in the Title I Education Center under the director of compensatory services. The Center was chosen because it had the expertise to get SES up and going most quickly and easily. The director of compensatory services is responsible for building SES into Toledo's strategic plan, and she also works directly with eligible schools. But the complexity of the SES endeavor led the district to also appoint a full-time NCLB facilitator to supervise day-to-day SES operations.

Anticipating that some of its schools would have SES-eligible students, the district did not wait for official notification to start moving on SES. Toledo carefully reviewed NCLB regulations and laid the groundwork for implementation. Making decisions about where to locate the program, how to staff it, and how to integrate it into existing district operations was an important early step. The distirct's Title I leaders also made sure that principals and teachers understood the law and its implications for eligible students. All these preliminary actions made for a smooth implementation process once official notice came from the state.

SES staff also worked with the district's Business Affairs Department to develop a comprehensive "Supplemental Educational Services Agreement" identifying the roles and responsibilities of the provider and of the district. Central to that agreement is the Student Learning Plan, which describes learning objectives, the timeline for meeting them, performance measures, how parents, school district staff, and the state department of education will be kept apprised of the student's progress, and the timetable for improving the student's achievement.

Once eligible students were identified, Title I staff began notifying parents by mail, informing them that they could obtain free tutoring for their children and giving them a brochure that briefly describes the services of each of the 14 providers from whom their child can get tutoring, including the district's own Reading Academy program. The mailing included a request-for-services form in English and Spanish, and the materials urged parents to select a provider and return the form by the deadline in order to enroll their children in the SES program of their choice. Fewer than 30 parents out of 1,500 whose children were eligible for SES responded to this first mailing.

The lesson was quickly driven home that the district needed to take a more personalized approach. Deciding to capitalize on parents' connection to their children's school and classroom, district SES staff met with the principals and Title I coordinators from the targeted schools to create a parent outreach campaign. The district kept schools informed about who had and had not enrolled for SES services, and principals and teachers began calling or meeting with the parents to encourage them to register their children. In addition, staff from the Title I Education Center set up information tables with flyers and applications in English and Spanish during parent conference periods, parent meetings, and other school events. With some SES experience behind them, satisfied parents have also begun spreading the word. With participation numbers increasing (from 96 the first year to over 500 the second), Toledo's SES staff identifies school-based marketing as an essential strategy for SES success.

^*Toledo's public school population is 46 percent African American, 44 percent white, and 7 percent Hispanic, with 2 percent representing other groups.

The project methodology is an adaptation of the four-phase benchmarking process used by the American Productivity and Quality Center, ^* along with case descriptions of individual districts and cross-site analysis of key findings. Although classic benchmarking looks for best or promising practices, using quantitative measures and comparisons among organizations, the practice of implementing supplemental educational services programs is too new to fully support this methodology. A brief description of this project's adapted methodology follows.

Plan

First, a conceptual framework was developed from an analysis of research on out-of-school programs and organizational management as well as an examination of what districts need to do to meet the No Child Left Behind Act's supplemental educational services requirements. Researchers, providers, and district administrators recruited to serve on an external advisory panel (see page 43) provided feedback to refine this framework and set priorities for issues to investigate. The resulting study scope guided all aspects of the study (see figure 16).

Site selection was a multistep process to ensure that the guide would feature an array of practices covering the elements of the framework and would represent a variety of geographic locations and contexts with which district administrators could identify. A list of 36 potential supplemental educational services sites was compiled through primary and secondary research by WestEd with suggestions from the expert advisory panel. A screening template was developed to systematically analyze the weighted criteria for site selection identified by the advisors, including the presence of clearly articulated strategic plans for administering SES, outreach and communications strategies for both parents and providers, and explicit contracting and recordkeeping procedures The template was completed for the candidate districts based on public documents such as brochures, reports, and district Web sites, supplemented by targeted phone interviews with district staff. The five districts that were selected had relatively high ratings on the template for preliminary evidence that promising practices were in place. No site was uniformly excellent, but each had developed practices in several areas from which others might learn.

Collect Data

Collecting detailed descriptive information from project participants was key to understanding the district's practices, the outcomes or impact achieved, and lessons learned in implementation from which others could. benefit. The major steps to this phase were finalizing the site-visit interview guide and arranging and conducting site visits to the innovation sites.

Each of the five innovation sites hosted a two-day site visit that included interviews with providers, principals, and parents, as well as district staff, to obtain multiple perspectives on the questions in the site-visit discussion guide. In addition, artifacts from the sites, such as letters to parents or brochures or contracts, were collected to provide concrete examples of district practices. The study team collated the information collected during the site visits and developed a case study for each site.

Analyze and Report

Once all the information was collected, the project team analyzed the data to understand the promising practices uncovered throughout the benchmarking project, both within and across sites. Thirteen key findings discussed in the final report emerged from the cross-site analysis.

Two products resulted from this research: a report of findings and this practitioner's guide. The report of findings provides an analysis of key findings across sites, a detailed profile of each site, a collection of artifacts, and key project documents. The practitioner's guide is a shorter excerpt and summary that is intended for broad distribution. The guide and report are also accessible online at http://www.ed.gov/nclb/choice.

Adapt

Ultimately, readers of this guide will need to select, adapt, and implement practices that meet their individual needs and contexts. Dissemination will take place through a variety of channels. The guide will be broadly distributed around the country through presentations at national and regional conferences, as well as through national associations and networks.

Districts coming together in learning communities may continue the study, using the ideas and practices from these sites as a springboard for their own action research. In this way, a pool of promising practices will grow, and districts can support each other in implementation and learning.

^*American Productivity and Quality Center. (2001). Benchmarking in education: Pure and simple. Houston, TX: Author.

General

American Association of School Administrators (AASA). AASA compiles resources and best practices to support implementation of NCLB, including relevant articles and organizations. http://www.aasa.org/nclb

Council of Chief State School Officers (CCSSO). CCSSO's Web site includes a section on NCLB, highlighting a variety of resources, including the SEA Toolkit on Supplemental Educational Services (http://www.ccsso. org/Federal_Programs/nclb/3349.cfm). The Toolkit provides suggested criteria, tools, and advice for state education agencies to use in approving supplemental educational service providers. http://www.ccsso.org

Early Implementation of Supplemental Educational Services Under the No Child Left Behind Act: One Year Report. This April 2004 report by the U.S. Department of Education offers an in-depth look at early SES implementation through case studies of six states and nine school districts. The study intention ally included states and districts that were relatively far along in their implementation; so while they are not representative of implementation nationwide, they offer useful insights. The report is available online: http://www.ed.gov/about/offices/list/ous/ppss/reports.html#title

Harvard Family Research Project (HFRP) Out-of-School Time (OST) Program Evaluation Database. This database is a searchable compilation of HFRP-written profiles of evaluations of OST programs and initiatives. It provides accessible information about evaluation work on both large and small OST programs to support the development of high-quality evaluations and programs in the out- of-school time field. http://www.gse.harvard.edu/hfrp/projects/afterschool/evaldatabase.html

National Governors Association (NGA). The NGA has an initiative on "Extra Learning Opportunities," including tutoring. The NGA's Center for Best Practices provides assistance to states and publishes resource materials. http://www.nga.org/center/topics/1,1188,D_363,00.html

National Institute on Out-of-School Time (NIOST). NIOST provides information and research reports on out- of-school time, including an annual Fact Sheet. It also provides training for directors and staff, school administrators, community leaders, and others committed to -providing high-quality after school programs for children and youth. http://www.niost.org

Supplemental Educational Services Quality Center. This project of the American Institutes for Research provides technical assistance at the local and state level through a network of demonstration districts and states The Center Web site will present "lessons learned," tools, and other resources for effective supplemental services implementation. The Center also facilitates a national network of organizations to coordinate and improve research and assistance on supplemental services.http://www.tutorsforkids.org

Supplemental Educational Services Webcast. This series of Web-based video segments about SES implementation was developed to inform the staff of state and local education agencies who guide and support the work of schools. Produced by the U. S. Department of Education, the segments include an overview of SES implementation and a panel discussion with three SES practitioners. http://www.ed.gov/print/admins/comm/suppsvcs/seswebcast.html

Resources for Engaging Parents

Black Alliance for Educational Options (BAEO). Founded in 2000, BAEO is a national, nonprofit organization that actively supports parental choice to empower families and to increase educational options for black children. BAEO has been funded by the U.S. Department of Education to develop a public information campaign to reach parents about the choices available to them under NCLB, and it currently works through 30 local chapters in 20 states. http://www.baeo.org

Committee for Hispanic Children and Families, Inc. (CHCF). The organization was founded in 1982 by a group of Latino health and human service professionals to improve the quality of life for Latino children and families and their communities. Its mission is to help educators and health professionals communicate with Latino parents in culturally sensitive ways. The Committee develops and implements outreach programs for low-income families and children in youth development and education. http://www.chcfinc.org

Education Resource Organization Directory (EROD). Districts can search this large, U. S. Department of Education-sponsored database to find local agencies and organizations whose purpose is to create connections with parents and inform them about educational options. http://wdchqdcolp02.ed.gov/Programs/EROD/

Greater Educational Options (GEO) Foundation. GEO was founded in 1998 with the goal of making educational choice a reality through extensive community outreach and educational awareness programs. Focusing primarily on Indiana and Colorado, GEO maintains a Web-based parent information and resource center.http://www. geofoundation.org

Hispanic Council for Reform and Educational Options (CREO). Founded in 2001, CREO is dedicated to improving educational outcomes for Hispanic children by empowering families through parental choice in education. Hispanic CREO has parent organizers on staff. In 2004, with funding from the U.S. Department of Education, it created Project CREO 2004 to reach out to Latino parents in five urban communities with high concentration of Hispanics (Austin, Dallas , and San Antonio, Texas; Miami, Florida; and Camden, New Jersey) for the express purpose of raising parents' awareness about NCLB's school choice and SES provisions. http://www.hcreo.org/

The National Coalition for Parent Involvement in Education (NCPIE). Founded in 1980 as an advocacy group for parents, NCPIE also offers a database of resources and organizations that support outreach to parents. Coalition members include parent organizations, foundations, and national education groups representing teachers and administrators. The coalition monitors legislation, initiates projects, and shares information and ideas about parent involvement in public education. NCPIE's recent publication, Helping Parents & Your Community Better Understand No Child Left Behind, is a set of action briefs that provide useful information about the major themes covered in NCLB in parent-friendly language. http://www.ncpie.org

Parent Information Resource Center (PIRC). Funded through a discretionary grant program of the U.S. Department of Education Office of Innovation and Improvement (OII) under Parent Options and Information, PIRCs have operated since 1995 to support parent involvement in Title I schools. Their mission is to provide parents, schools, and organizations with training, information, and technical assistance to understand how children develop and what they need to succeed in school. More than 70 PIRCs across the country work closely with parents, educators, and community-based organizations to strengthen partnerships that support children in reaching high academic standards. PIRCs are ideally positioned to help school districts get the SES word out to parents of eligible students. http://www.pirc-info.net

U.S. Department of Education, Office of Innovation and Improvement (OII). OII's Web site includes pdf versions of an SES poster and brochure that districts can tailor for their own use. http://www.ed.gov/about/offices/list/oii/about/index.html

1. The providers who had identified themselves as willing to work in one or more of the districts at the time of this study were: A+ Educational Centers, ABC-Learn, Inc., Acadamia.NET, LLP, Baden Street Settlement, Beyond the Bell Learning Centers of LAUSD, Boys and Girls Clubs of America, Brainfuse, Club Z! In-Home Tutoring Services, Dial-A-Teacher, Dream Builders, EdSolutions, Educational and Tutorial Services, Educational Enterprises, Failure Free Reading Instant Achievement Center, Fresh Start Academy, Huntington Learning Centers, Inc., Kaplan K-12 Learning Services, Kumon Math and Reading Centers, Learning for Life, Boy Scouts of America, Math*Ability, Monroe #1 BOCES, Nazareth College, Neighborhood Youth Association, New Life Learning Center, Newton Learning, Pacific Metrics Corporation, Paradigm Learning Center, Platform Learning, Princeton Review, Professional Tutors of America, Progressive Learning, Project IMPACT, PSI Affiliates, Inc., Reading Academy of Toledo Public Schools, Reading and Language Arts Centers, Inc., Reading Revolution, San Diego State University, San Diego Unified School District, Say Yes to Life, SCORE! Educational Centers, Smart Kids Tutoring and Learning Center, Inc., SMARTHINKING, Inc., Specialized Student Services, Inc., Sylvan Education Solutions, Sylvan Learning Systems, Inc., The Talking Page Literacy Organization, Tutors Of The Inland Empire, and Ventures Education Systems.

2. Another guide in this series covers creating a strong district school choice program.

3. As explained in the U.S. Department of Education’s Supplemental Educational Services Non-Regulatory Guidance (2003, August 22, p. 17), an LEA might be unable to offer choice within the district if all of its schools at the relevant grade level are in school improvement or it has only a single school at that grade level. Also, choice may be impractical if the district’s other schools are located at a large distance (e.g., 100 miles) from the school in improvement status. In such cases, LEAs must, if possible, enter into cooperative agreements with neighboring LEAs (or with charter or “virtual” schools in the state) that can accept their students as transfers. When such options are not possible or practical, LEAs are encouraged to offer SES to students attending schools in their first year of improvement.

4. This is the range from the lowest 10^th percentile to the highest 90th percentile in district per-pupil funding level in the states in which the five study districts are located. Figures are drawn from the ESEA Title I LEA Allocations, FY 2003, as posted at: http://www.ed.gov/about/overview/budget/titlei/fy03/index.html

5. One resource for states as they screen providers is the SEA Toolkit disseminated by the Council of Chief State School Officers. This toolkit provides approval criteria, tools, and advice for state educational agencies to use as they approve supplemental educational service providers. http://www.ccsso.org/content/pdfs/SSPToolkit.pdf

6. One resource for districts in working with service providers is the Guide to Working with Model Providers in comprehensive school reform. This guide advises schools on the stages involved in the development of effective partnerships from the negotiation of a contract, to the end of the contract. http://www.centerforcsri.org/pubs/ModelProviders.pdf

7. U.S. Department of Education. (2003, August 22). Supplemental Educational Services Non-Regulatory Guidance, p. 13.

8. Ibid, p. 5.

9. Ibid, p. 27.