Administration on Aging (AoA)

AoA Program Evaluations and Related Reports

AoA program evaluations insure that: the most relevant data are available to policy makers; programs demonstrate value to the taxpayer; and programs have a track record of results. AoA strives to evaluate programs in an integrated manner combining process, outcome, impact and cost-benefit analysis of evaluation activities. This site provides links to reports and results from these evaluation efforts.

Completed Evaluations and Studies

Evaluability Assessment of the American Indian, Alaska Natives and Native Hawaiian Programs (Title VI)

In 2014, ICF International (ICF) was contracted by ACL/AoA to conduct the Evaluability Assessment (EA) of the Title VI Grant Program. The purpose of the contract was to provide ACL with:

A description of the program model (What do grantees do, and how do they operate?)
An assessment of how well defined the programs/program services are (Are the programs/program services stable, distinct, consistent and established?)
Information about what Federal and Tribal stakeholders want to get from an evaluation and how the evaluation data findings would be used.

The study examined the program characteristics of Title VI grantees’ nutritional, supportive, and caregiver support services to assess the feasibility of, and best approaches for, formal evaluation of the Title VI Program. The Evaluability Assessment of the Title VI Grant Program Final Report (PDF, 1.26MB) includes program background, recommendations for enhancing Title VI grantees’ readiness for evaluation, and evaluation design recommendations. A comprehensive program logic model and a medicine wheel model are included.

Evaluation of the Aging and Disability Resource Centers (ADRCs)

There are more than 550 ADRC sites across the nation. Since the inception of this initiative, ADRCs have had over 4.8 million contacts with consumers, caregivers, providers and professionals. The focus of the evaluation is to determine the extent to which Aging and Disability Resource Centers (ADRCs) are fulfilling the goal of improving awareness of and access to long term supports and services for older adults and individuals with disabilities and also how well ADRCs are contributing to the overall ACL mission. The process data collection, completed in the fall of 2013, had high response rates at 100% of State-level ADRCs and 84% of local ADRC grantees. The outcome study data collection was completed in the spring of 2014 with more than 600 respondents from 33 sites. Data from a presentation (PDF, 2.23MB) about the process evaluation is available as is the final study report (PDF, 5.85MB) with appendices (PDF, 9.84MB). The final report provides background on the ADRC program, study methodology and results for both the process and outcome studies. The results of this evaluation will influence future performance measures and indicators. This work was completed by IMPAQ International with Abt Associates under contract HHSP233201000692G.

Process Evaluation of the Chronic Disease Self Management Education Program (CDSME)

In September 2011, AoA awarded Contract HHSP233201100492G to IMPAQ International and Altarum Institute. This process evaluation examined state CDSMP programs funded through Communities Putting Prevention to Work: Chronic Disease Self-Management Program, an initiative of the Administration for Community Living/Administration on Aging (AoA) in collaboration with the Centers for Disease Control and Prevention (CDC) and the Centers for Medicare & Medicaid Services (CMS). The research team employed a multi-method approach using multiple qualitative and quantitative data sources. The program data were also used to conduct regression analyses to examine the influence of various factors on participant completion rates. The report provides information about: characteristics of State grantees, CDSMP participants, CDSMP implementation, participant completion rates, site-level data collection, program sustainability, and recommendations for program improvements. The final process evaluation report (PDF) with Appendices (PDF) is now available. In addition to the process evaluation, ACL is coordinating with CMS’ efforts in evaluating evidence-based wellness programs. A copy of the CMS report which includes CDSMP outcomes is available at innovation.cms.gov (PDF). For additional information about this grant program please also see the reports created by ACL’s technical assistance grantee the National Council on Aging at www.ncoa.org/improve-health/center-for-healthy-aging/capping-reports.html.

Real People Real Problems: An Evaluation of the Ombudsman Programs of the Older Americans Act (The National Academies Press)

2004 Survey of Adult Protective Services: Abuse of Adults 60 Years of Age and Older (PDF) (National Center on Elder Abuse)

Title III-B Supportive Services Evaluation

Final Report: DOC | PDF
Interim Report of Quantitative Analysis (PDF)

Ongoing Studies

Evaluation of Title III-C Elderly Nutrition Services Program (ENSP)

The Administration for Community Living (ACL) is conducting a three-part evaluation of its Title III-C Nutrition Services Program (NSP). Part one is a process evaluation that provides information to support program planning by analyzing program structure, administration, staffing, coordination, and service delivery as well as the interactions between the many levels and types of organizations that provide congregate meals, home-delivered meals, and collateral services under the Title III-C NSP.Part two is a cost study that estimates the costs of program operations, the most important being the cost of the congregate and home-delivered meals provided using Title III funds, and to examine cost variation within the program by cost component and program characteristics. And part three assesses program effectiveness, as measured by the program's effects on a variety of important outcomes (including nutrient adequacy, socialization opportunities, and health outcomes and, ultimately, helping older adults avoid institutionalization) through comparing program participants' outcomes with those from a matched comparison group of eligible nonparticipants. Although the client outcome study is still underway, the Process Evaluation Report and the Cost Study Report are available now. In addition, you can watch a briefing on these reports or download the briefing handouts (PDF).

The Caregiver Outcome Evaluation Study of the National Family Caregiver Support Program

The National Family Caregiver Support Program (NFCSP) represents a significant Federal investment in supporting caregivers who provide care and assistance to aging adults and to grandparents raising grandchildren. Through this program, the Aging Network helps meet the immediate needs of caregivers and care recipients while also being the catalyst for broadening the long-term care (LTC) service systems at State, Territory, local, and Tribal levels to better support families. Its ultimate goal is to support individuals who prefer to age in their own homes and communities—as opposed to institutional settings—through lower-cost, nonmedical services and supports. The National Family Caregiver Support Program, authorized by Title III-E of the Older Americans Act (OAA), provides information to caregivers about available services, assistance to caregivers in accessing supportive services, individual counseling, support groups and caregiver trainings, respite care, and supplemental services.

ACL is currently conducting a two-part evaluation of the NFCSP program. The first part, conducted by the Lewin Group, was a process evaluation with the overall purpose of understanding and documenting the strategies used to meet NFCSP goals. This part was completed in March 2016 and produced a briefing of the results (PDF, 2.72MB), an executive summary of the results (PDF, 1.21MB), and a final process evaluation report (PDF, 2.31MB). In addition, State Units on Aging were asked to submit their assessment tools. The evaluation team has compiled them and grouped them into the following categories:

If you encounter access issues with this material, please email Susan Jenkins for additional assistance.

The second part of the evaluation will be conducted for the Administration for Community Living by Westat, a nationally-recognized research firm from 2016 through 2018. This part will focus of people who receive the services and answer the following questions:

Do NFCSP program participants deal better with emotional, physical and financial tolls of caregiving than non-NFCSP caregivers?
Do NFCSP program participants have less depression, anxiety, and stress attributable to their caregiving than non-NFCSP caregivers?
Do NFCSP program participants report providing care longer than non-NFCSP caregivers?
Do NFCSP services enable care recipients to remain in their homes and communities when that is the family’s desired outcome?

If you have questions about this evaluation, please email Susan Jenkins for additional assistance.

Evaluation Design Projects

Evaluation Design for the Long Term Care Ombudsman Program (LTCOP)

In September 2011 the Administration for Community Living (ACL) awarded contract HHSP233201100500G to NORC at the University of Chicago (NORC) to develop an evaluation study design to better understand and assess the effectiveness of Long-Term Care Ombudsman Programs (LTCOPs). The task included building the evidence base on LTCOPs in order to develop recommendations for a rigorous and comprehensive study design that investigates program efficiency and program effectiveness at multiple levels, including the resident/family, facility, local/state/program, and federal levels. Key tasks of the design process involved the development of a family of four logic models and a set of overarching research questions to guide the evaluation, as well as the identification of data collection tools and data sources that inform those questions. The diversity of proposed activities reflects the ACL’s goals for this evaluation, the commitment to a population health frame of reference, and seven critical LTCOP characteristics that influence design options. The final evaluation design report (PDF, 2MB) was completed in January 2013.

Evaluation Design for a Global Evaluation of Older American Act (OAA) Services

Under this project, the Administration on Aging (AoA) seeks to study the impact of OAA programs and services, specifically services provided under Title III, on key outcomes including HCBS use, health care use, community tenure, and long-term services and supports (LTSS) expenditures. OAA services and programs are diverse, often integrated and/or provided in combination with other services, funded through multiple funding streams, and administered and delivered by different state and local-level agencies with varying data collection capacity. This design focuses on the impact of services provided through OAA funded HCBS programs authorized under Titles III-B, C, D, and E of the OAA. These programs include a range of supportive services, nutrition services, health promotion and disease prevention programs, as well as services for family caregivers. While the mix and type of services offered differ by state and locality, the vast majority of OAA funding is used for the provision of nutritional services. Proposed research questions include:

What is the impact of OAA funded HCBS programs and services on:
- Community tenure
- Health care utilization
- Costs of care for older adults (e.g., LTSS, health care costs)
- Physical, mental, and emotional health and wellness (e.g., preventive measures) of care recipients and caregivers
- Unmet needs among older adults
- Caregivers (e.g., strain, burden, depression, health, etc.)
- Coordination of services (e.g., care management)?
What is the impact of OAA services alone or in combination with services paid for by other sources?
What is the impact of service mix and intensity on outcomes of interest? [if possible, we will isolate OAA services]
What subgroups had the most favorable outcomes? (e.g., health conditions, demographics, functional status)

A primary recommendation from the final evaluation design report (PDF) is that the Administration for Community Living (ACL) first facilitate an exploratory study to determine the feasibility and limitations of conducting this evaluation with state OAA data. Such a study will assist ACL in identifying actions needed to better position the states for participation in a nationwide evaluation.

Last Modified: 8/26/2016