Program Performance Monitoring and Benchmarking Expected Levels of Achievement at Specific Time Intervals Within a Program Year

"If you can't see success, you can't learn from it or reward it. If you can't recognize failure, you can't correct for it." Organizations that measure the results of their work--even if they do not link funding or rewards to those results--finds that the information transforms them."

Introduction

To reach GEAR UP's goals, GEAR UP partnerships and staff must be committed to continually measuring the progress of students and the impact of resources and services provided. In an initial review of the newly revised work plans submitted by the grantees, the GEAR UP staff found that the objectives needed to be more specifically stated, the measurable outcomes needed to be aligned to those objectives, and benchmarks needed to be established. Thus, it was clear that the grantees needed technical assistance in formative evaluation.

In order to continually measure changes in students' academic performance, parents' aspirations for their children to attend college, and teachers' expectations for low income students to complete rigorous courses, it is essential to first examine baseline data on GEAR UP participants. These data show students' demographic characteristics and other socioeconomic factors, school attendance patterns, enrollment in rigorous academic courses, academic achievement, promotion and retention rates, graduation rates, as well as the extent of parental involvement in their children's education. Overall, the data should reflect the current status of students based on various criteria. By examining these data, students' strengths and weaknesses can be assessed, and appropriate GEAR UP objectives, services and activities, and realistic outcomes for a program's first year can be determined.

1. Benchmarking Expected Levels of Achievement in GEAR UP Programs

   As the GEAR UP programs evolve and additional early intervention strategies are implemented, it is essential for grantees to monitor their program performance and students' progress in improving school attendance and classroom achievement. Thus, for each annual objective, grantees can establish "benchmarks," which are expected levels of achievement at specific time intervals during the school year. Establishing benchmarks allows GEAR UP schools to make progress towards achieving an objective, gradually, steadily, and continuously over several time intervals within a year. Each benchmark represents a somewhat higher expected level of performance than the previous benchmark at an earlier point in time. "Performance benchmarking" is the continuous rigorous practice of comparing actual outcomes at specific time intervals within a budget year, against/relative to established standards. By comparing expected to actual outcomes, you and Department of Education staff can know whether or not students, parents, and teachers are making sufficient progress in the mastery of a particular subject area, skill, behavior, or other desired outcome.

   Performance benchmarks are intended to reflect, for instance, the growth of students' academic performance and understanding of college program and financing options. Suppose a specific objective for a 7^th grade cohort is "50 percent of Prealgebra students will be ready to enter and succeed in Algebra I by June 2000 (up from 35 percent of the previous year's 7^th grade Prealgebra class)." Potential benchmarks can be: "By November 1999 and February 2000, 40 percent and 50 percent of students, respectively, will perform at least at the 'B' level"; and "By April 2000, 50 percent of students will score within the highest two stanine groups on the annual standardized math test."

   Similarly, if an objective related to parental involvement states, "At least 75 percent of cohort students' parents will attend the monthly evening parent workshops," then by December 1999, 25 percent of parents can be expected to attend; by March 2000, 50 percent can be expected to attend; and by June 2000, 75 percent are expected to attend the workshops. Additionally, if an objective related to teachers' expectations is that "By June 2000, at least 80 percent of middle school teachers will concur that low income students can be motivated to pursue and successfully complete a rigorous college preparatory curriculum," then it is possible to expect that by December 1999 and March 2000 that 40 percent and 60 percent of teachers, respectively, will believe in low income students' potential to complete a rigorous academic program.

2. Planning and Implementing a GEAR UP Program with a Measurable Impact

The following set of questions were developed to guide you and your partners in determining that what you are doing is effective, promoting significant growth in student academic achievement and parents' involvement in their children's education, raising teachers' expectations about low income students' potential to enter and successfully complete college, and producing measurable outcomes.

1. What objectives are we focusing on this year so that the broad goal of GEAR UP is ultimately achieved?
2. What types of activities, services, and other intervention strategies can be offered that are directly linked to the objectives? How frequently should such provisions be offered (e.g., daily, weekly, semi-monthly?) and for how long (one hour before or after school? one-half day on selected Saturdays?) Are any particular cohort students (e.g., students with a specific skill deficiency, career interest) targeted for certain GEAR UP services? What level of expertise, experience, and training should be required of the individuals providing specific services?
3. Why are we choosing to implement these particular intervention strategies and offer these activities? How will low-income youths be more motivated once this particular service is available? What evidence do I have that a particular activity, such as "tutoring," will address the cohort's educational needs and enhance their capacity for more accelerated learning? Moreover, how will GEAR UP services in particular (as compared to the school's current initiatives) improve disadvantaged students' classroom achievement, standardized test scores, and ability to meet more rigorous graduation requirements?
4. What impact can I expect the supportive services to have on low-income youths' classroom performance and achievement? Given the baseline data and the programs we are implementing, how much improvement can we expect in students' attitude toward learning about and preparing for the future; attendance patterns at school and class; development of sufficient skills to enter and succeed in Algebra I by the end of 7^th grade; and knowledge of college programs, costs, and financing opportunities? How much increase in parents' participation in school programs (such as monthly Saturday activities, monthly college preparation workshops, and biweekly counseling sessions) is realistic? What percentage increase in attendance should we expect in weekly "brown bag lunches" for the cohort's teachers?
5. What strategies should I use to determine if we are making sufficient progress toward meeting the objectives that were established at the beginning of the year? How do I determine that what I am doing is working, promoting students’ acceleration, and producing a measurable outcome? What procedures should I use to measure the impact of GEAR UP on at-risk middle school youths' preparation to potentially attend college? How should I assess changes in students' attitude toward preparing academically for college and planning for a productive future? How do I verify that parents are becoming more actively involved in their children's school programs and helping them plan financially and academically for college? What are some valid, research-based approaches to measuring changes in GEAR UP teachers' expectations about low income students' capacity to prepare for, enter, and complete college?
6. Finally, what evidence do we have that our program is or is not succeeding? Should we change the type, format, frequency, or duration of any particular activity? What should we be doing differently?

TSDS Home