Detailed Information on the
Enhancing Education Through Technology Assessment

Collect data on annual, long-term, and efficiency performance measures and use these data to establish baselines and performance targets.

Make program performance information available to the public in a transparent manner.

Work with States to ensure program goals and purposes are understood, data reporting is accurate, and stakeholders are constantly working to improve program performance.

Measure: Percentage point difference in Internet access between classrooms in high- and low-poverty schools.

Explanation:High-poverty schools are defined as having 75 percent or more of students eligible for free or reduced -priced lunch, and low-poverty schools are defined as having 35 percent or fewer students eligible for free or reduced-priced lunch. There is no statistically-significant difference between the two groups in 2002 and 2003.

Measure: Percentage of districts receiving program funds that have effectively and fully integrated technology.

Explanation:Each State, by statute, is required to establish its own definition for technology integration.

Measure: Percentage of teachers, in districts that receive program funds, that meet State technology standards.

Explanation:

Measure: Percentage of eighth-grade students that meet their State's technology literacy standards.

Explanation:EETT authorizing statute requires all students to be technology literate, as defined by their State, by the end of eighth grade.

Measure: The percentage of monitoring reports that the Department of Education sends to States within 45 days of an Educational Technology State Grants monitoring visit (both on-site and virtual).

Explanation:The measure was changed from number of days to percentage of time that the Department was able to meet a set standard of 45 days, which provides a more effective measure of program management and efficiency.

Measure: The percentage of States that respond satisfactorily within 30 days to findings in the Department's monitoring reports.

Explanation:The measure was changed from number of days to percentage of time that it takes for States to meet a set standard of 30 days, which provides a more effective measure of program management and efficiency.

Is the program purpose clear?

Explanation: The purpose of the Department of Education's (ED) Enhancing Education Through Technology (EETT) program is to improve student academic achievement through the use of technology in schools. This program addresses the purpose through direct grant support to State Education Agencies (SEAs), which then make subawards to Local Education Agencies (LEAs) through formula allocations based on population as well as through grant competitions.

Evidence: Title II, part D of the Elementary and Secondary Education Act, as amended by No Child Left Behind.

Does the program address a specific and existing problem, interest, or need?

Explanation: The optimal usage and distribution of educational technology are significant issues, and schools have achieved progress in these areas. In 2003, 93 percent of schools had access to the internet in their classrooms, and the ratio of students to computers was 4.4 to 1. The National Education Technology Plan reports that, while there is access to computers and the internet, there is still a lack of adequate training and understanding of how computers can be used to enrich the learning experience.

Evidence: Internet Access in U.S. Public Schools and Classrooms: 1994-2003; Toward a New Golden Age in American Education, National Education Technology Plan 2004, U.S. Department of Education.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: EETT is the only large-scale Federal program dedicated to address hardware and technology training needs in classrooms. Program funds can be used for a range of technology-related activities, from acquiring hardware to providing professional development to teachers. The other large-scale Federal technology program, E-rate, only provides support for internet connectivity and internal connections at eligible schools and libraries (as well as phone services), and does not allow for the funding of other activities supported by EETT.

Evidence: The State Educational Technology Directors Association recently reported that 24 percent of State technology directors cited EETT funds as the only source of funds the State educational agency awards to districts specifically dedicated for technology, and an additional 50% cited EETT as the primary source of funds.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: States are required to distribute half of their EETT funding by formula and half through a competition. While States can ensure that a limited number of districts receive substantial grant awards through the competition, many districts receive a formula award inadequate to affect significant change.

Evidence: For example, in FY 2003, the average award to districts eligible for the Small, Rural School Achievement program was $2,173. States reported, on average, that a formula allocation below $21,000 was considered insufficient for purposes of targeting competitive funds.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Despite the varied award amounts under the ETTT formula, the formula is tied to districts' shares of Title I funds. Therefore, districts with greater proportions of high-need students receive larger amounts of funding. Further, the competitive funds are statutorily required to be targeted to high-poverty districts.

Evidence: Section 2411 (a) (2) of the Elementary and Secondary Education Act, as amended by No Child Left Behind.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: ED recently developed two long-term measures for this program: (1) The percentage of ninth-grade students who have met State technology literacy standards (2) In districts that receive funding from the State Grant program, the percentage of teachers that meet their State's technology standards.

Evidence: Currently, 18 States have stand-alone technology standards for students, 16 report that technology standards are integrated within other academic standards, and 8 report having both stand-alone and integrated standards. Only two States formally assess the technology skills of students, and an additional 11 report having plans to do so. Twenty-seven States have technology standards for teachers, and 5 formally assess teacher progress in the use of educational technology.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: ED has set an ambitious goal that all teachers meet State technology standards and all students are technologically literate by 2014. Interim targets will be established after baseline data are reported in fall 2006.

Evidence:  

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: ED has established the following annual performance measures for this program: (1) The percentage of districts receiving EETT funds that have effectively and fully integrated technology, as identified by States (2) The percentage difference in Internet access between classrooms in high- and low poverty schools. Further, ED recently established two efficiency measures to assess the operational efficiency of State monitoring visits. ED and OMB will continue to work on establishing an efficiency measure more closely related to program outcomes.

Evidence: Data for the first measure will be collected through ED's EDEN data initiative, and baseline data will be available in fall 2005. Data for the second measure are collected through the annual NCES survey, Internet Access in U.S. Public Schools and Classrooms. Baseline data for the efficiency measures will be available in Fall, 2006.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: ED has collected information on Internet access in classrooms, and nationally States and school districts have made progress in narrowing the access gap between high- and low-poverty schools in the last ten years. ED expects to have baseline data for the integration and efficiency measures in the fall of 2006. ED will establish targets once baseline information is available.

Evidence: The 2004 target is to eliminate the gap in classroom Internet access between high- and low-poverty schools.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Through the Consolidated State Performance Report, which includes specific questions about EETT, ED has confirmed grantee commitment to working toward the program's goal of improving student achievement through the use of technology in schools. States and districts are implementing strategies to integrate technology into classroom instruction, largely through professional development.

Evidence: All States and the Bureau of Indian Affairs have submitted their Consolidated Performance Reports.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: ED is currently conducting an independent evaluation of ETTT. The evaluation examines program implementation and educational technology usage in States, districts and schools receiving funds from this program. A final report is expected in late 2007.

Evidence: The National Educational Technology Trends Study.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The 2006 Budget did not request any funding for EETT. Although this request is tied to the accomplishment of a major policy goal (that of eliminating funding for programs that do not reflect an appropriate Federal role or, for other reasons, are not Administration priorities), it has not been tied to accomplishment of the program's annual or long-term performance goals. However, the ED's budget submissions detail the full cost (including S&E) for all programs.

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ED is working to ensure that high -quality data are available for performance measure reporting. Data are collected through both an independent evaluation and annual grantee performance reports. The quality of grantee reporting is not consistent, however, and ED is taking steps to provide technical assistance to the States to improve reporting quality. Further, ED has held meetings with State educational technology directors to provide technical assistance and clarify the importance of complying with the evaluation requirements of the program.

Evidence: Annual Performance Reports, Meetings with stakeholders.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ED regularly collects high-quality performance data for ETTT through surveys, the national evaluation, and the annual performance reporting requirements. These data are used to identify where grantees are not making progress and accordingly target technical assistance and informational activities.

Evidence: Two of ED's primary data sources, the national evaluation and the survey of internet access, are subject to the evaluation standards of the independent evaluators and the National Center for Education Statistics, respectively. ED recently hosted an institute for State educational technology directors to address the evaluation requirement in response to State concerns with this challenge.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Program managers are held accountable through Education Department Performance Appraisal System (EDPAS) plans, which directly link employee performance to relevant EETT program and Strategic Plan goals. Specifically, staff are held accountable to meet essential deadlines and commitments, particularly pertaining to grants, monitoring for performance results and outcomes, and annual performance reports. Additionally, program staff are required to annually submit a monitoring plan, grant investment portfolios, and grants schedules which holds managers accountable to ensuring grantee success. Monitoring plans include how programs will assess GPRA measures and high-risk situations. Additionally, States are held accountable annually through reporting requirements in the Consolidated State Performance Report.

Evidence: EDPAS agreements; ED monitoring plan; Grant Schedule; Consolidated State Performance report

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds are obligated within the timeframes set by ED schedules and used for purposes intended. ED reserves some funds for program evaluation, which are obligated based on an evaluation plan. Additionally, ED reserves some funds for national activities, including the development of the National Education Technology Plan, which are obligated based on an annual, OMB-approved, spending plan.

Evidence: Grantees are drawing down funds at an acceptable rate.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ED has recently adopted two efficiency measures to assess the operational efficiency of State monitoring visits. Shortening the time between the visit and ED's initial report as well as between ED's report and the State's response will ensure that technical assistance, guidance, and other next steps will respond to immediate needs and allow for a more efficient use of time and resources.

Evidence: Baseline data for the efficiency measures will be available in Fall, 2006.

Does the program collaborate and coordinate effectively with related programs?

Explanation: EETT collaborates with all programs subject to the transferability provision of the No Child Left Behind Act (21st Century Community Learning Centers, Improving Teacher Quality State Grants, Safe and Drug-Free Schools State Grants, and State Grants for Innovative Programs). EETT staff have collaborated with the other programs regarding technology integration, necessary application and reporting requirements for districts that transferred funds, and to promote effective ways that technology can facilitate the achievement of program goals. Also, the EETT and Title I Grants to LEAs offices recently held a conference call with 33 States to facilitate communication between State Education Technology and Title I Directors. Additionally, program staff collaborated with the ED's Strategic Accountability Service to develop and administer a FY 2005 Customer Satisfaction Survey to gain feedback from State Education Technology Directors. Lastly, EETT staff are collaborating with the FCC to develop performance measures and guidance on technology plans for the E-rate program.

Evidence: FY 2005 Customer Satisfaction Survey of State Education Technology Directors; Updated regulations for the E-rate program; Federal Communications Commission 04-190, pgs 19-23.

Does the program use strong financial management practices?

Explanation: Recent agency-wide audits have not identified deficiencies in the financial management of this program.

Evidence:  

Has the program taken meaningful steps to address its management deficiencies?

Explanation: While material internal management deficiencies have not been identified for this program, EETT has implemented a system to identify potential problems. Additionally, program staff have instituted performance measures that assess the primary statutory goals of the program, and have taken steps to ensure that quality data are collected and reported through both evaluation and State reporting.

Evidence: Program staff monitor excessive and deficient drawdowns of funds to prevent high-risk situations; Office of Elementary and Secondary Education Risk-Abatement Matrix; National Education Technology Trends Study; GPRA measures.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: ED maintains information on grantee activities through consolidated annual reports, site visits and compliance monitoring, and technical assistance activities.

Evidence: ED monitors program activities through virtual/ videoconferencing, phone conferencing, and site visits. ED also conducts interim review and assessment meetings, where State Directors brief ED on progress made and outcomes from EETT funded interventions. Additionally, program staff regularly participate in National and regional conference presentations by grantees that showcase the grantees' activities.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: ED is collecting a series of data related to this program, including the EETT evaluation, annual surveys on connectivity and distance learning, and a longitudinal study of the impact of technology in schools. Interim and final reports of program and technology-level research are available to the public through ED Pubs. Additionally, ED's "EDEN" data initiative includes a number of State -level data elements on educational technology. This data will be made available to the public, and will be disaggregated at the State level.

Evidence: National Education Technology Trends Study; U.S. Department of Education, National Center for Education Statistics. Distance Education at Degree-Granting Postsecondary Institutions: 200-2001, NCES 2003-17, by Tiffany Waits and Laurie Lewis. Project Officer: Bernard Greene. Washington, DC: 2003; Parsad, B., and Jones J. (2005). Internet Access in U.S. Public Schools and Classrooms: 1994-2003 (NCES 2005-015). U.S. Department of Education. Washington, DC: National Center for Education Statistics.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Data is not yet available to measure performance toward the longterm performance goals.

Evidence: ED expects to have baseline data for the measures in 2006.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The EETT performance target of equal classroom Internet access between high- and low-poverty schools was met, according to 2003 data. Baseline data for the other long-term measure (percentage of districts that have fully and effectively integrated technology) will be available in fall 2005.

Evidence: In 2003, 95 percent of classrooms in low-poverty schools and 90 percent of classrooms in high-poverty schools reported having Internet access. While there is still a modest gap, it is not considered statistically significant.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: While ED has established two operational efficiency measures for this program, baseline data is not yet available for setting targets or determining efficiency. ED and OMB will continue to work on establishing an efficiency measure more closely related to program outcomes.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The E-rate program is the only other Federal program that supports investments in educational technology of a similar scale as EETT. However, E-rate funds can only be used for Internet connectivity and internal connections (as well as phone services) at eligible schools and libraries while Education Technology program funds may be used for other technology activities, such as professional development and curriculum design. Given the wide divergence in allowable activities, there is no clear way to compare the performance of the two programs.

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Preliminary evaluation data show that States are using EETT funding in accordance with NCLB, while adopting state-wide technology plans and strategies.

Evidence: 96% of States reported that their technology plan includes a long-term strategy to improve student achievement through the use of technology, and 79% indicated that their technology strategies and goals align with academic standards. Further, States set priorities for their competitive awards that are aligned with program goals-supporting professional development (90% of States), integrating technology into the classroom (80%), and improving student academic achievement in core subject areas (75%).