| Name of Program or Resource | Type of Program or Resource/Year Initiated | Type of Evaluation Featured in Guidea | Year Evaluation Started | Evaluation Objective |
|---|---|---|---|---|
| Alabama Connecting Classrooms, Educators, & Students Statewide Distance Learning | Online courses and interactive video-conference classes for students across state/Piloted in spring 2006; statewide implementation in fall 2006 | External; formative & summative; includes comparisons with traditional instructional settings | 2006 | Monitoring program implementation; program improvement; sharing best practices |
| Algebra I Online | Online algebra courses for students across the state/2002 | External and internal; formative and summative; includes comparisons with traditional instructional settings | 2003; comparative study in 2004-05 | Determine if program is effective way to provide students with certified algebra teachers and to support the in-class teacher's certification efforts |
| Appleton eSchool | Online courses for students enrolled in district's high schools (some students take all courses online)/2002 | Internal; formative; evaluation process based on internally developed rubric | Rubric piloted in 2006 | Program improvement; sharing best practices |
| Arizona Virtual Academy | Virtual charter school for students enrolled in public schools and homeschool students (no more than 20%)/2003 | Formative and summative; external and internal | 2003 | State monitoring; quality assurance; program improvement |
| Chicago Public Schools - Virtual High School | Online courses for students enrolled in district's high schools/2002 | External; formative | 2002 | Assess need for mentor training and other student supports; identify ways to improve completion and pass rates |
| Digital Learning Commons | Web site with online courses and a wide array of resources for teachers and students/2003 | External and internal; formative | 2003 | Understand usage of site; assess impact on student achievement and college readiness |
| Thinkport - Maryland Public Television with Johns Hopkins | Web site with a wide array of resources for teachers and students/2000 | External and internal; formative and summative, including randomized controlled trial | 2001; randomized controlled trial in 2005 | Understand usage of site; assess impact of "electronic field trip" on student performance |
| Cost of Evaluation | Funding Source for Evaluation | Data Collected | Data Collection Tools | Improvements Resulting From Evaluation |
|---|---|---|---|---|
| $60,000 in 2007; $600,000 in 2008 | Specific allocation in program budget (originating from state of Alabama) | Student enrollment, completion, grades; APb course pass rates; student and teacher satisfaction; description of implementation and challenges | Surveys, interviews, observations | Teacher professional development; improvements to technology and administrative operations |
| $110,000 for the most labor-intensive phase, including the comparative analysis during 2004-05 | General program funds, grants from NCREL,c BellSouth Foundation, and U.S. Department of Education | Student grades and state test scores; pre- and posttests; student use and satisfaction data; focus groups; teacher characteristics and teachers' certification outcomes | Pre- and posttests developed by evaluator, surveys | Teacher professional development; increased role for in-class teachers, curriculum improvements, new technologies used; expansion to middle schools |
| No specific allocation; Approx. $15,000 to make the rubric and evaluation process available in Web format | General program funds (originating from charter grant from state of Wisconsin) | Internal descriptions and assessments of key program components (using rubric); mentor, student, and teacher satisfaction data; course completion and grades | Internally developed rubric and surveys | Mentor professional development; course content improvements; expanded interactivity in courses; improvements to program Web site and printed materials; sharing of best practices |
| No specific allocation | General program funds (originating from state of Arizona) | Student enrollment, grades, and state test scores; parent, teacher, and student satisfaction data; internal & external assessments on key program components | Electronic surveys; externally developed rubric | Wide range of operational and instructional improvements |
| Approx. $25,000 | District's Office of Technology Services | Student enrollment, course completion, grades, and test scores; student use and satisfaction data; mentor assessments of needs | Surveys, interviews, focus groups | Designated class periods for online learning; more onsite mentors; training for mentors |
| Approx. $80,000 for the college-readiness study | Bill & Melinda Gates Foundation | Student transcripts; student grades and completion rates; use and satisfaction data | Surveys | Improvements to student orientation; curriculum improvements; development of school use plans to encourage best practices |
| Estimated $40,000 for the randomized controlled trial (part of a comprehensive evaluation) | Star Schools Grant | Student test scores on custom-developed content assessment; information about delivery of curriculum; use and satisfaction data | Test of content knowledge developed by evaluator, surveys, teacher implementation logs | Changes to teaching materials; changes to online content and format |
a See Glossary of Common Evaluation Terms on page 65.
b Run by the nonprofit College Board, the Advanced Placement (AP) program offers college-level course work to high school students. Many institutions of higher education
offer college credits to students who take AP courses.
c North Central Regional Educational Laboratory.
Back to Evaluating Online Learning