Psychological Health and Traumatic Brain Injury Program Evaluation & Improvement
Program evaluation is an important part of the DCoE mission. It includes efforts to help military program administrators and leadership conduct evaluations that assess the operation and performance of their programs, by examining program structure, processes, results and financial costs to determine how the program meets required needs.
Program evaluation, if incorporated into the regular operational activities of military programs, is an important tool that enables continuous improvement and helps achieve better service quality and outcomes. When the routine measurement and assessment of program quality and performance becomes a regular activity across the Defense Department, we create a culture of effectiveness that can better address the needs of service members and their families.
To learn more, click on the links below:
What drives program evaluation across the military?
Government stakeholders tasked DCoE, under key directives, to conduct program evaluation and improvement (PEI) activities across psychological health and traumatic brain injury (TBI) programs within the Defense Department. Through these directives, DCoE executes this mission on behalf of the Military Health Service and service-specific leadership by providing information and tools that enhance the effectiveness of psychological health and TBI programs. Below are some of these key directives:
-
The White House:
Presidential
Executive Order No. 13625
for 2012, “Improving Access to Mental Health Services for Veterans, Service Members and Military Families”
-
Office of Management and Budget:
DoD
agency priority goal
for fiscal year 2013, to, “improve effectiveness of behavioral health programs”
-
Congress:
National Defense Authorization Act for fiscal year 2013,
section 739, “Plan to eliminate gaps and redundancies in programs of the Department of Defense on psychological health and traumatic brain injury”
DCoE supports a culture of effectiveness across the Defense Department and promotes evaluation as an inherent component of new and existing psychological health and TBI programs. Our PEI capability will inform programmatic decision-making and policy recommendations, in addition to guiding efforts to improve existing programs.
What is the DCoE program evaluation and improvement capability?
DCoE implements a PEI capability across psychological health and TBI programs. DCoE and its component centers use this capability to track and maintain information on program effectiveness and to identify trends, gaps and promising practices to help benefit other programs. As the Defense Department’s designated agent for determining the effectiveness of psychological health and TBI programs, DCoE conducts program evaluation studies; provides support to individual programs through training, tool kits and additional services; and submits reports to stakeholders. A key DCoE goal is to develop strong and lasting partnerships with programs and stakeholders and to serve as a trusted advisor and collaborator on matters related to program effectiveness across the Defense Department.
DCoE seeks to incorporate regular program evaluation into every new and existing psychological health and TBI program. In doing so, DCoE provides a platform to build strong, reliable and effective prevention and care programs across the Defense Department.
The DCoE vision and mission for program evaluation and improvement support an organizational goal to serve as a central facilitator for psychological health and TBI program effectiveness.
-
Vision:
To improve the effectiveness of psychological health and TBI programs and treatment outcomes for service members and their families.
-
Mission:
To be the principal authority and trusted source on psychological health and TBI program effectiveness in order to inform senior-level decision-making about psychological health and TBI programs across the Defense Department.
-
Value Proposition:
To develop the core capability for evaluating psychological health and TBI programs and services for service members and their families.
How does the DCoE program evaluation and improvement capability work?
To support enhanced program effectiveness across the military, DCoE provides program assistance functions in the form of
program evaluation
and
change management. Program assessment
is the building block of those functions and the critical first step in understanding program needs and offering adequate support.
Program Assessment:
The initial step in the program evaluation and improvement process is program assessment. This involves gathering basic program information and conducting baseline assessments across military psychological health and TBI programs to determine readiness for full evaluation. Program assessment will provide information on potential gaps and redundancies across the system. Depending on where a program falls within the evaluation readiness continuum, DCoE determines how best to support the program through more extensive program evaluation studies and/or through change management activities.
Program Evaluation:
When a program is ready for a full evaluation, DCoE conducts a comprehensive program evaluation study to understand how the program functions, its short-term and long-term impact, and the overall costs of the program. DCoE develops evaluation reports to provide to programs and higher-level stakeholders. These reports will summarize strengths and opportunities for improvement. This information guides development of training, tool kits and support strategies to help programs improve their effectiveness. Additionally, as systematic evaluations occur, trends and promising practices are identified and shared, thereby making DCoE a repository for information and resources related to effective psychological health and TBI program practices.
Change Management:
DCoE offers change management support to promote organizational transformation and data-driven decision-making, with program evaluation as a necessary component of every program. Through change management, DCoE helps ensure that all psychological health and TBI programs are evaluation-ready. This is accomplished by providing training, tool kits, communications and support services designed to assist programs in producing accurate and reliable information needed to monitor improved effectiveness. Outreach support focuses on a variety of topics such as program design, measurement strategies, data collection, data analysis and process improvement.
What are the benefits of building a culture of effectiveness across the Defense Department?
The ultimate objective of DCoE program evaluation effort is to build a culture of effectiveness across Defense Department psychological health and TBI programs. Some of the anticipated benefits include:
-
Ready access to tools and other resources to enhance program effectiveness:
Psychological health and TBI programs now have access to program evaluation tools and individualized support. These resources are designed to help make incorporation of regular program evaluation activities and the transition to a culture of effectiveness, less burdensome, so programs can continue to focus on the task of serving military members and their families.
-
Increased self-understanding:
Programs will have accurate up-to-date information about their operations, outcomes and overall effectiveness. This information informs efforts to improve outcome performance and provides objective information to facilitate higher-level decision-making to enhance effectiveness across the system. Understanding and addressing gaps and redundancies promotes a better use of resources to get the best services to the right people at the right time.
-
Quick identification and use of promising practices:
DCoE’s program evaluation and improvement activities make information about promising practices and emerging trends more available to providers, program administrators and Defense Department leadership, which helps increases system effectiveness.
-
Data-driven decision-making processes:
Incorporating regular program evaluation activities to induce a culture of effectiveness provides consistent, high-quality information to guide decision-making by Defense Department leadership. Consistent use of program evaluation methods ensure that all stakeholders have a similar understanding of what an effective program looks like and that decisions about funding, staffing and program selection are driven by objective data.
Resources
DCoE Program Evaluation Guide 2nd Edition
Training:
For upcoming webinars and training events, please visit the DCoE Webinar page. You can also access archived webinars on the Program Evaluation and Improvement Archived Training webpage or watch videos on the DCoE YouTube channel.