Detailed Information on the
IDEA Special Education - Parent Information Centers Assessment

Develop a strategy for evaluating the impact and effectiveness of program activities.

Develop a baseline and targets for the program's efficiency measure.

Assess the data and methodologies used for annual, long-term, and efficiency measures to determine their reliability as indicators of performance.

Develop baselines and targets for three new measures that have been adopted for the Parent Information Centers program.

Develop baselines and targets for the program's 2 long-term measures.

Conduct a data based analysis of the distribution of funds among parent centers to determine the most effective allocation of funds.

Measure: The percentage of materials used by PTI projects that are deemed to be of high quality.

Explanation:Data for the measure will be gathered through an expert panel review of materials used by projects.

Measure: The percentage of products and services deemed to be of high relevance to educational and early intervention policy or practice by an independent review panel of qualified members of the PTI target audiences.

Explanation:Data for the measure will be gathered through an expert pannel review of products and services provided by projects.

Measure: The percentage of all products and services deemed to be useful by target audiences to improve educational or early intervention policy or practice.

Explanation:Data for the measure will be gathered through an expert panel review of products and services provided by projects.

Measure: The cost per output, by category, weighted by the expert panel quality rating.

Explanation:Data for the measure will be gathered through an expert panel review.

Measure: The percentage of parents receiving PTI services who promote scientifically- or evidence-based practices for their infants, toddlers, children and youth.

Explanation:Data for the measure will be gathered through a sample survey of parents receiving Parent Information Centers services. Data will be collected biennially.

Measure: The percentage of parents receiving PTI services who report enhanced knowledge of IDEA rights and responsibilities.

Explanation:Data for the measure will be gathered through a sample survey of parents receiving Parent Information Center services. Data will be collected biennially.

Is the program purpose clear?

Explanation: The purpose of the program is to provide training and information to parents of children with disabilities on their rights and protections under the Individuals with Disabilities Education Act (IDEA) so that they can develop the skills necessary to participate effectively in planning and decisionmaking relating to early intervention, educational and transitional services, and in systemic-change activities. Centers also help parents understand the nature of their children's disabilities and needs so that they can help improve their children's education and life outcomes.

Evidence: IDEA Part D sections 681(b)(1), 682(b) and 683(b).

Does the program address a specific and existing problem, interest or need?

Explanation: Parental involvement and advocacy are critically important to the development and education of children with disabilities. Because IDEA services, procedures and protections are complicated, parents need specialized skills and knowledge to address issues where training and information are not readily available from other sources.

Evidence: Parents are automatic participants in the development of their children's Individualized Education Programs (IEPs) and Individualized Family Service Plans (IFSPs) and need expertise in a wide range of areas ranging from the evaluation of children's needs to the identification of appropriate services and educational goals. (See IDEA Part B section 614(d) and Part C section 636).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program supports (1) parent training and information centers and (2) community parent resource centers. Parent training and information centers generally serve whole States or large portions of States. Community parent resource centers are in smaller, local areas within the larger areas covered by parent training and information centers. They focus especially on the needs of underserved parents. There is no significant overlap with other Federal programs, though many States provide additional support for parent information activities. Parents may also receive assistance relative to specific disabilities from non-profit associations focused on the needs of children with those particular disabilities (such as the ARC for children with mental retardation).

Evidence: IDEA Part D sections 682(b)(1) and 683((a).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: We do not have evidence that another approach, mechanism, or infrastructure would be more efficient or effective to achieve the program's purposes.

Evidence:  

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Awards for both parent training and information centers and community parent resource centers must be made to parent organizations as defined in IDEA, whose members are highly motivated and focused on addressing parent needs. In addition, the program supports technical assistance activities so that centers have the knowledge and capacity to provide quality services to parents. However, the community parent resource centers are able to serve only a small percentage of underserved parents. Also, while community parent resource centers may be effective in reaching their target populations of underserved parents, these populations are typically relatively small. Compared with parent training and information centers, the cost of serving these parents is relatively high.

Evidence: Defintion of "parent organization" is in IDEA Part D Section 682(g). Technical assistance is required in Section 684. For 2003, $2.6 million (10%) of total program funds was for technical assistance. In 2003, Education provided $20.7 million for 72 parent training and information centers and $3 million for 30 community resource centers.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program does not have meaningful long-term measures. The Department is also working with OMB on developing an appropriate efficiency measures.

Evidence: Lack of long term measures.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program does not have meaningful long-term measures or ambitious targets.

Evidence: Lack of long term goals.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Program staff recently participated in Department-wide meetings to develop common measures for assessing the performance of ED technical assistance programs. The data for measures generated through these meetings will be collected in 2006. Implementation includes development of a methodology for convening of panels of scientists and practitioners to review products and project designs and developing an instrument for obtaining data from target audiences on the usefulness of ED TA products and services.

Evidence: Draft Common Measures for Education Technical Assistance Programs.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: OSEP still needs to develop baselines and targets for these annual measures.

Evidence: Draft Common Measures for Education Technical Assistance Programs.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Parent organizations are typically highly motivated and focused on addressing parent needs. In the past, they have worked towards OSEP's performance goals for all PART D program and are likely to be committed to the new annual goals.

Evidence: Parent Centers Helping Families Data Outcomes (1997 - 2002) Report.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: There has not been an independent, high quality evaluation of the program, but OSEP is planning one in 2004 or 2005. Nevertheless, the program monitors and assesses grantee performance and for many years, grantees have systematically collected a wide range of useful program management data. While the objectivity of these self-assessments may be questionable, the information provided does help OSEP staff support program improvements.

Evidence: Parent Centers Helping Families Data Outcomes (1997 - 2002) Report.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Budget request is not tied to either annual or long-term goals.

Evidence: Department of Education Fiscal Year 2005 Budget.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Although OSEP has been working to address its strategic planning deficiencies, meaningful actions to eliminate such deficiencies have not yet been implemented. As OSEP works to address planning deficiencies, it is placing particular emphasis on "adopting a limited number of specific, ambitious long-term performance goals and a limited number of annual performance goals."

Evidence: The program is participating in Education's Technical Assistance common measures group but more work still needs to be done to correct strategic planning deficiencies.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: All parent center grantees are required to submit Annual Performance Reports and Final Reports and OSEP staff work closely with them to review these reports. A team of reviewers also conducts an indepth assessment of the Technical Assistance grantee in its second year. However, it is unclear how information gathered in such reports translate into improved performance/accountability for grantees, or how they are linked to more management initiatives to better allocate resources or adjust program priorities.

Evidence: 3+2 evaluation of the Alliance Project (OSEP's TA contractor for parent centers) and grantee annual performance reports.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Currently, ED cannot demonstrate how federal managers and program partners are held accountable for program goals. However, the Department has initiated several efforts to improve accountability in its programs. First, ED is in the process of ensuring that EDPAS plans -- which link employee performance to relevant Strategic Plan goals and action steps ' hold Department employees accountable for specific actions tied to improving program performance. ED is also revising performance agreements for its SES staff to link performance appraisals to specific actions tied to program performance. Finally, ED is reviewing its grant policies and regulations to see how grantees can be held more accountable for program results.

Evidence: The President's Management Agenda scorecard (Human Capital and Budget & Performance Integration initiatives) notes ED's efforts to improve accountability. ??The Department's Discretionary Grants Improvement Team (DiGIT) recommendations indicate that ED is reviewing its grant policies and recommendations.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: OSEP successfully obligates funds by the end of each fiscal year and on a timely basis. Funds are spent for the intended purposes and no improper uses of funds have been identified.

Evidence: Finance reports, notices of competitions, lists of funded applications.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: To date, the Department has not established procedures for this program to measure and achieve efficiencies in program operations. However, ED is in the process of developing its competitive sourcing Green Plan, and is working to improve the efficiency of its grantmaking activities. The Department has also established a strengthened Investment Review Board to review and approve information technology purchases agency-wide.

Evidence: Department Investment Review Board materials. ED's Discretionary Grants Improvement Team (DiGIT) recommendations.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Education has convened a technical assistance working group to better coordinate similar TA&D programs in OSEP, IES, the What Works Clearinghouse, and elsewhere. All programs will collect common annual performance measures starting in 2006 on program quality, relevance, and utility. Also, OSEP is working to ensure that its various TA&D project grantees are collaborating with each other on program activities and strategies in order to reduce duplication.

Evidence: Application notices. www.ed.gov/legislation/FedRegister/announcements/2004-2/042104i.pdf. Example Web site for Kentucky can be seen at www.kyspin.com/. Web sites for all parent centers can be accessed through the National Dissemination Center for Children with Disabilities at www.nichey.org. Select "State Resources", Choose a State, "Parent Organizations".

Does the program use strong financial management practices?

Explanation: Auditors have not reported internal control weaknesses. The Department has a system for identifying excessive draw downs, and can put individual grantees on probation where draw downs need to be approved.

Evidence:  

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has taken steps to address some of its management deficiencies. For example, the President's Commission on Special Education identified the "peer review" process as an area of weakness in current program management practices. In response, OSEP has provided Internet training on the peer review process. However, OSEP's inability to produce a Comprehensive Plan as required by the IDEA Amendments of 1997 for this and other Part D National Activities program remains a major problem.

Evidence:  

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: New awards are based on a clear competitive process.

Evidence: OSEP application notices.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: OSEP reviews awardee performance through annual performance reports and final reports, and holds annual meetings with project officers in Washington. When necessary, OSEP staff also conduct site visits to review grantee activities.

Evidence: Annual performance reports and OSEP's "3+2" evaluation for project providing TA to centers.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Performance data is collected annually from awardees. However, these data are not readily available to the public in a transparent and meaningful manner. Education is developing a department-wide approach to improve the way programs provide performance information to the public. In 2004, Education will conduct pilots with selected programs to assess effective and efficient strategies to share meaningful and transparent information.

Evidence: Lack of transparent data for the public.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program does not have meaningful long-term measures.

Evidence: Lack of long term measures.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program still needs to develop annual performance goals.

Evidence: Lack of annual performance goals.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Department is working with OMB on developing an appropriate efficiency measure for this and other Education TA&D programs.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: We have no systematic evidence to compare OSEP's Parent Centers program with other TA&D programs. However, the Department is currently working with OMB to develop a limited number of cross cutting performance indicators that may allow for such comparisons in the future.

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: There has not been an independent evaluation of this program, but OSEP is planning an evaluation of all of its Part D National Activities in 2004 or 2005.

Evidence: Lack of an independent program evaluation.