DEVELOPMENTAL DISABILITIES PROGRAM INDEPENDENT EVALUATION (DDPIE) PROJECT

What is the Developmental Disabilities Program Independent Evaluation (DDPIE) Project?
The DDPIE project is an independent (non-biased) evaluation of the impact of the state Developmental Disabilities (DD) Network programs 1 on the lives of people with developmental disabilities and their families. The intent of this evaluation is to understand and report on the accomplishments of these programs, including collaborative efforts among the DD Network programs. The results of this evaluation will provide the Administration on Developmental Disabilities (ADD) (the agency that administers these programs) with information to judge the effectiveness of its programs and policies and serve as a way for ADD to promote accountability to the public.

Why is the evaluation being conducted?
The independent evaluation is a response to accountability requirements for ADD. The need for accountability is identified in the Developmental Disabilities Assistance and Bill of Rights Act of 2000 (DD Act), the Government Performance and Results Act (GPRA) of 1993, and the Program Assessment Rating Tool (PART), administered by the Office of Management and Budget (OMB). This project meets the requirements of PART by providing a non-biased method of evaluating the effectiveness and impact of DD Network programs in serving people with developmental disabilities and their families.

How is this project conducted?
The DDPIE project is divided into two phases. Phase I consists of two activities: (1) the development of tools for determining the impact of DD Network programs on people with developmental disabilities and their families; and (2) a pilot study to test the accuracy, feasibility, and utility of these tools. Phase II of the evaluation will involve a full scale independent evaluation of the DD Network using the tools designed in Phase I.

Westat, a private research firm in Rockville, Maryland, has been engaged by ADD to conduct Phase I of the independent evaluation (September 30, 2005 - September 29, 2008). Throughout this project, Westat has attempted to involve DD Network stakeholders in the process of developing tools for the independent evaluation. Efforts have also been made by both Westat and ADD to provide ongoing information on progress so the process of tool development can be as transparent as possible. Westat has worked with representatives from DD Network programs who assisted Westat in characterizing the key functions of each program. This project also has an Advisory Panel that provides on-going advice on the design and implementation of Phase I of the independent evaluation. In addition, a Validation Panel will be used to review results of the pilot study and provide recommendations for the methodology and implementation of data collection tools. Both panels consist of people with developmental disabilities, family members, other consumers, advocates, researchers, representatives from the DD Network, and policy and research specialists.

What is the progress to date?
During the first year of the project, Westat focused on becoming familiar with the three DD Network programs by reviewing background materials and conducting interviews with key informants. By the end of the first year, Westat developed a framework for measurement matrices with assistance from Working Groups. In Year 2, the major activities included the development of the measurement matrices (consisting of key functions, benchmarks, indicators, and examples of performance standards for the DD Network programs). These measurement matrices were further refined based on input and feedback received from various stakeholders (e.g., DD Network program staff, Advisory Panel members). These draft measurement matrices were then used to guide the development of data collection instruments for the pilot study.

What are the next steps?
Westat is in the process of conducting the pilot study in order to test the logistics and data collection instruments developed for the independent evaluation.  The pilot study will consist of data collection from ten selected DD Network programs (i.e., four DD Councils, three P&As, and three UCEDDs). Westat randomly selected all programs for the pilot study within pre-set criteria, which included factors such as size, geographic location, and organizational structure 2. In addition, the pool from which to sample excluded programs based on factors such as previous involvement with this project 3. The pilot study will consist of in-person individual and group key informant interviews and individual telephone interviews.

After the pilot study, Westat will then review, summarize, and synthesize collected data and present results and recommendations to the Validation Panel meeting in June, 2008. Finally, as of September, 2008, Westat will write and submit a final report to ADD based on recommendations received from the Validation Panel and results derived from the pilot study.

Where can I get more information about this project?
If you have additional questions about the evaluation or pilot study, please feel free to contact:

1. Lynn Elinson, PhD, Project Director, Westat, lynnelinson@Westat.com
2. Jennifer Johnson, EdD, Project Officer, ADD, jennifer.johnson@acf.hhs.gov