Jo Anne B. Barnhart
Commissioner of Social Security

Inspector General

Performance Measure Review: The Social Security Administration’s Transition Planning (A-02-01-11014)

Following consultations with congressional committees, the Office of the Inspector General agreed to review the Social Security Administration’s performance indicators over a continuous 3-year cycle. We recently completed our first 3-year cycle. In conducting this work, we used the services of an outside contractor, PricewaterhouseCoopers, LLP (PwC), to assist us in our efforts.

For this report, we used PwC to conduct the review of two of the Agency’s performance indicators related to its transition planning. The objective of the review was to assess the reliability of the data used to measure the Agency’s transition planning efforts.

Please comment within 60 days from the date of this memorandum on corrective action taken or planned on each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at (410) 965-9700.

James G. Huse, Jr.

This report is one of five separate stand-alone reports, corresponding to the following Social Security Administration (SSA) process and performance measures (PM):

• Create Agency change strategy. (PM #9)

Fiscal Year (FY) 2000 Goal: Develop and implement change strategy.

• Complete Agency plan for transitioning to the workforce of the future. (PM #10)

This report reflects our understanding and evaluation of the process related to PMs #9 and #10. To achieve its strategic goal, "To be an employer that values and invests in each employee," SSA has developed several strategic objectives. Two of these objectives are 1) "Promote an Agency culture that successfully incorporates our values," and 2) "Create a workforce to serve SSA’s diverse customers in the twenty-first century." SSA’s FY 2001 Annual Performance Plan (APP) contains two performance indicators developed to meet this objective as follows:

• Create Agency change strategy - Created to meet SSA objective to, "Promote an Agency culture that successfully incorporates our values." This indicator will be considered achieved if SSA develops and implements a strategy for change.

• Complete Agency plan for transitioning to the workforce of the future - Created to meet SSA objective to "Create a workforce to serve SSA’s diverse customers in the twenty-first century." This indicator will be considered achieved if SSA completes an employee survey and publishes an Agency workforce transition plan.

We performed our testing from September 21, 2000 through February 15, 2001. Our engagement was limited to testing at SSA’s headquarters in Woodlawn, Maryland. The procedures that we performed were in accordance with the American Institute of Certified Public Accountants’ Statement on Standards for Consulting Services, and are consistent with appropriate standards for performance audit engagements in Government Auditing Standards (Yellow Book, 1994 version). However, we were not engaged to and did not conduct an audit, the objective of which would be the expression of an opinion on the reliability or accuracy of the reported results of the performance measures evaluated. Accordingly, we do not express such an opinion.

SSA's FY 2000 APP contains performance indicators related to its strategic goal to be an employer that values and invests in each employee. The resulting strategic objective is to promote an agency culture that successfully incorporates SSA values. The performance indicator, designated to track organizational progress made towards the goal of valuing and investing in each employee, is the creation of a change strategy that would instill the desired values. Nine desired values are listed in SSA’s FY 2001 APP as follows:

• To treat every individual with equal regard and respect;

• To act with integrity;

• To imbue quality in all we do and help others do the same;

• To remember that courtesy and compassion mean everything;

• To listen carefully and to act on what we hear;

• To respect stability and embrace change, keeping what works and changing what could work better;

• To value diversity;

• To discharge faithfully our role as guardians of the public trust; and

To fulfill this strategic goal and objective, SSA needed to ascertain the current values and culture of SSA and determine if gaps existed between the current values and the desired values. The Office of Workforce Analysis (OWA) within the Office of Human Resources (OHR) was tasked with developing a plan to achieve this objective.

OWA developed a plan that included four major process steps as follows:

Below is a brief description of the four major process steps taken by SSA to achieve its goal of developing and implementing an agency change strategy.

To perform a baseline study of the agency culture, SSA researched the topic of organization culture based on well-known and respected sources in the field. In addition, SSA followed standard qualitative data collection methods in order to develop an approach to discovering its culture and validating findings.

SSA also hired two external consultants to help it establish what it would need to accurately document its own culture. This research enabled SSA to define the meaning of culture for its own organization and to determine the particular constructs it needed to measure to obtain an accurate assessment of its culture.

Based on this research SSA identified three core cultural themes:

• Building confidence in SSA;
• Providing world-class service; and
• Creating a supportive work environment.

Appropriately, the OWA team then presented these three core themes to individuals and groups of SSA employees and managers for validation. The findings from these employee interviews, focus groups, and document reviews, formed the basis for the development of survey questions.

To conduct the analysis, SSA developed an employee survey. Employees were asked to rate their level of agreement with 67 statements related to workplace behaviors and practices (Section A), values (Section B), and work climate (Section C) for the current SSA environment and also how important the behavior or practice was to them. The survey included a demographics section that allowed the results to be cross-tabulated by region, regional component, grade level, years of service, age, gender, etc.

Section B specifically asked respondents to give their opinion on the nine values listed in the 1997 Agency Strategic Plan (ASP). The inclusion of these nine values was especially important, since it was the first time SSA employees had been asked about their opinion of the ASP values. The inclusion of these values allows SSA to build a more direct measurement from the ASP, APP, and the Government Performance and Results Act of 1993, in general.

The survey was administered by mail and sent to a randomly selected 20 percent of the total SSA employee population (13,600). A cover letter was included with the survey from the Deputy Commissioner of Human Resources. It stressed that responses would be kept confidential and that an outside group would be collecting and tabulating the survey results. The survey was also sent to all 128 members of the SSA Senior Staff. The response rate for the employee survey was 48.5 percent and 51 percent for the Senior Staff.

Survey results were analyzed by taking the means of each current and desired workplace statement and identifying the top ten highest ratings, lowest ratings, and the gaps between the two groups. The employee, mid-level manager, and Senior Staff responses were also compared.

The following items were identified as the top ten gaps:

• Gap 1: Having an appraisal system that supports clear and challenging performance standards.

• Gap 2: Rewarding individuals for good work.

• Gap 3: For employee opinions to count.

• Gap 4: Having job advancement opportunities available.

• Gap 5: Sharing work responsibilities fairly.

• Gap 6: Effectively recruiting and selecting new employees who are motivated and well-matched to their jobs.

• Gap 7: Treating every individual with equal regard and respect.

• Gap 8: Rewarding good teamwork.

• Gap 9: Delivering training in a timely and effective manner.

• Gap 10: Resolving conflicts in positive ways.

A benchmarking study was conducted to see how other organizations carry out a culture change effort. Thirty-seven organizations were selected for the benchmarking study (25 private and 12 public organizations). Eleven organizations ultimately participated in the study by responding to a survey; four of the eleven allowed SSA to conduct site visits. A conference call was conducted with a fifth benchmark partner. The results of the benchmarking study provided SSA with comparisons and ideas that could be implemented for their own culture change effort.

SSA found that most Government agencies responding to their best practice survey were not willing to share the details of their culture change efforts and found that those that did showed little evidence of change.

SSA formed an intercomponent workgroup (OCW) to examine the results of the culture survey and formulate a change strategy plan. The group included members from all relevant SSA stakeholders. While the 10 gaps found between the current and desired SSA cultures formed the basis for SSA’s change strategy plan; OCW also examined other sources of information relevant to the assessment of SSA culture as follows:

• The results of the 1999 SSA employee survey conducted by the National Partnership for Reinventing Government;

• Other current SSA initiatives that will influence organizational culture, such as the 2010 Vision;

• The results of the employee interviews;

• The results of the benchmarking studies; and

• The results of brainstorming and discussion sessions.

The OCW analyzed and categorized the 10 gaps into 3 groups: performance management, communications, and empowerment. The recommendations were categorized as follows:

• Theme #1: Create a comprehensive performance management system.

• Theme #2: Promote greater communication throughout SSA.

• Theme #3: Empower Individuals/Offices/Components/Agency.

During the period of September 21, 2000 to February 15, 2001, we evaluated the current processes and controls, which support the FY 2000 SSA performance measurement process. In addition, we determined the accuracy of the underlying performance measure data. Our evaluation of the information provided by SSA management as of February 15, 2001 allowed us to determine that the reported FY 2000 results of the performance measure tested (shown below) were reasonably stated based on the methodology used by SSA.

OWA has completed the four major steps and identified short- and long-term recommendations for SSA to help achieve its goal of becoming an employer that values and invests in each employee. We evaluated the approach that SSA has taken to study its culture and found that SSA’s overall strategy to change its organizational culture is based on solid research principles. We determined that SSA undertook a thorough effort to fully understand the theory and practice behind the study of organization culture and to define the characteristics of its own culture.

In addition, we found that the employee survey SSA developed was based on strong research and literature related to the measurement of organizational culture and used appropriate survey development standards. However, we did note the following opportunities for improvement in SSA methodology:

1. SSA has not assigned specific responsibility for implementing recommendations.
2. Cultural themes are not a direct measure of the culture.
3. Survey sections were not directly "mapped" to SSA’s values.

The change strategy is embodied within OCW’s short- and long-term recommendations, but specific responsibility for implementing these recommendations has not yet been assigned. SSA stated that it is currently awaiting approval of the strategy from senior management. However, assignments have not been identified within the document. Not assigning specific recommendations to particular individuals or offices will result in a lack of accountability mechanisms within the organization to help support these change initiatives.

We found that the three cultural themes are consistent with the tenor of the nine APP values. However, the cultural themes are not a direct measurement of the culture. Specifically, it is not clear how the survey sections and subsequent questions are directly "mapped" or cross-walked to these three core cultural themes.

Sections A and C of the survey were not directly "mapped" to the APP values as found in Section B. If Sections A and C could be mapped by group or category to the nine APP values, it would allow SSA to build a more direct measurement from the APP and GPRA in general. By mapping or grouping the nine values against Section A and C, it would be possible statistically to conduct factor analysis and obtain factor groupings. This analysis would provide a way to measure the nine APP values and see how they group or correlate to Sections A and C statistically. This analysis would help SSA to further define the areas where they need culture change and action plans.

Our evaluation found that OWA has completed the implementation process of this performance measure. OWA completed four major steps of developing and implementing a strategy by performing a baseline study, a gap analysis, a benchmark study, and identified short- and long-term recommendations for SSA to implement. However, SSA has not assigned responsibility for implementing recommendations.

This way, SSA may be able to gain helpful information from other federal organization’s performance indicators monitoring cultural change.

SSA's FY 2000 APP contains performance indicators related to its strategic goal to be an employer that values and invests in each employee. The resulting strategic objective is to create a workforce to serve SSA’s diverse customers in the twenty-first century. The resulting performance indicator to track achievement towards this goal is to complete SSA’s plan for transitioning to the workforce of the future. The FY 2001 goal is to implement and update the transition plan and develop and implement action items from the employee survey.

To fulfill SSA’s Strategic Plan of creating a workforce to serve SSA’s diverse customers in the 21^St century, SSA developed a 5-year workforce transition plan. The plan was developed with input from all SSA components and its labor and management organizations. The plan was based on:

• What SSA expects to happen in the future; what the effects of the future will have on SSA’s workforce needs;

• What actions will be taken to address SSA’s workforce needs.

SSA based the development of its workforce plan on the following components:

1) projected attrition analysis, 2) predicted succession flow for major SSA occupations, and 3) factors to be considered for future updates. SSA performed a workforce analysis to ascertain its current capabilities and expected needs based on these three components. The expectations of the future came from well-respected and legitimate sources such as the Census Bureau, the Office of Personnel Management and the analysis of its own Agency history of personnel trends.

SSA identified five major issues to be addressed by its workforce transition plan:

1. SSA needs to improve the process for projecting its workforce needs and for planning how to meet them to allow adequate time for recruiting and training;
2. SSA needs to recruit new employees that reflect the diversity of its customers, with the necessary competencies and who will stay long enough to give a return on investment;
3. SSA needs to fully develop and utilize its employees and provide them with the necessary tools and training;
4. SSA needs to provide a work environment and an organizational culture that supports employees and enables them to provide world-class customer service; and
5. SSA needs to explore additional avenues for meeting its future workforce needs.

SSA’s plan was comprised of the following items:

• Description of workforce planning issue,
• Action items to address issues,
• Milestones to track progress of each issue,
• Lead office responsible, and
• Target completion dates.

Each of these items is appropriately explained and the milestones are appropriate actions needed to address each workforce planning issue. To increase manageability and accountability, SSA has assigned a lead office responsible for each action item.

SSA has recently administered a pilot of its employee survey (January 16-February 2, 2001) and the analysis of the survey data is in process. An official SSA report has not yet been released on the status of this strategic goal.

The purpose of conducting the employee survey is to assess current levels of satisfaction with job and work environment. Once the data is collected, SSA will have the knowledge to help make SSA a better place to work. The information gathered from the survey will help SSA to meet its Strategic Objective to "create a workforce to serve SSA’s diverse customers in the twenty-first century."

The intended steps for administering a pilot employee survey are listed below. SSA has completed the first five steps.

1. Provide a validated survey instrument grounded in sound statistical research;
2. Conduct a pre-survey publicity and communications campaign to maximize employee interest and participation;
3. Print and distribute all survey materials;
4. Administer the survey using a blend of machine scorable questionnaires, toll-free interactive phone responding, internet/intranet methodology and/or any other appropriate survey response mechanism;
5. Take appropriate steps to ensure a minimum response rate of 75 percent;
6. Conduct train-the-trainer sessions on how to teach managers/supervisors to analyze and use the survey findings for the development of action plans to improve satisfaction;
7. Provide an in-depth written analysis of the overall survey results with specific recommendations for next steps SSA should consider based on these results;
8. Conduct executive level briefings on overall findings;
9. Generate feedback reports to Headquarters covering consolidated results for the test sites;
10. Provide feedback reports to all survey participants covering the findings for their individual work units at the lowest level of supervision; and
11. At the conclusion of the full process test, SSA will conduct an internal evaluation to determine the advisability of proceeding with a full implementation, across SSA.

During the period of September 21, 2000 to February 15, 2001, we evaluated the current processes, systems and controls, which support the FY 2000 SSA performance measurement process. In addition, we determined the accuracy of the underlying performance measure data. Our evaluation of the information provided by SSA management as of February 15, 2001 allowed us to determine that the reported FY 2000 results of the performance measure tested (shown below) were reasonably stated.

We have evaluated SSA’s approach to creating its workforce transition plan and found that SSA has completed the plan and provided a well-thought out strategy to implement the transition plan and to meet its diverse workforce needs. We found that SSA performed its analysis based on standard workforce analysis procedures and reasonable assumptions of the future.

In addition, our evaluation of the questions and structure of the employee survey for the process test and have found it to be a well-researched and tested survey instrument. SSA has chosen a reliable metric that can be benchmarked and also has the option of adding three additional questions to the employee survey. We found that the development and implementation of the SSA employee survey process test is proceeding according to schedule and appears to be on-target for meeting its goal of improving SSA work environment.

Given the nature of long-range planning and unpredictability of the future, we found SSA’s approach to be reasonable.

The workforce plan describes the method SSA can use to achieve this strategic objective, but it is not appropriate for measuring if SSA has implemented its plan and is achieving the goals of the plan.

Of the 13 recommendations contained in this report, SSA agreed with 8, disagreed

with 1, and deferred comments on 4 of the recommendations pending Agency action on the OCW report. The Agency disagreed with the recommendation that it select a smaller sample size and expend more resources to obtain a high response rate for that smaller sample when surveying its employees on SSA’s culture. It deferred on four recommendations that called for changes to the questionnaire used to survey its employees. Please see Appendix C for the full text of the Agency’s Comments.

We appreciate SSA’s comments to this report. The implementation of the recommendations contained within this report will help to ensure for the better measurement of SSA’s culture and its ability to transition its workforce to meet the needs of citizens in the future. We continue to recommend that SSA revisit the size of the sample it used when surveying its employees on its culture. SSA had a sample of over 13,000 employees and a response rate below 50 percent. In future surveys, SSA should spend less of its resources on administering a very large sample of employees and more of its resources on ensuring that an adequate number of those sampled actually respond to the survey.

We understand the Agency’s desire to defer its response to four recommendations pending action on the OCW report. We will continue to track these recommendations and look forward to the Agency’s response to them in the near future.

APPENDIX A – Scope and Methodology

APPENDIX B – Acronyms

APPENDIX C – Agency Comments

APPENDIX D – Performance Measure Summary Sheets

APPENDIX E – Performance Measure Process Maps

APPENDIX F – Performance Measure Taxonomy

The Social Security Administration (SSA) Office of the Inspector General contracted PricewaterhouseCoopers, LLP (PwC) to evaluate 11 SSA performance indicators identified in its Fiscal Year 2001 Annual Performance Plan (APP). We performed our testing from September 21, 2000 through February 15, 2001.

Our engagement was limited to testing at SSA’s headquarters in Woodlawn, Maryland. The procedures that we performed were in accordance with the American Institute of Certified Public Accountants’ Statement on Standards for Consulting Services, and are consistent with appropriate standards for performance audit engagements in Government Auditing Standards (Yellow Book, 1994 version). However, we were not engaged to and did not conduct an audit, the objective of which would be the expression of an opinion on the reliability or accuracy of the reported results of the performance measures evaluated. Accordingly, we do not express such an opinion.

For these two performance indicators, PwC was asked to assess SSA’s progress to date in the implementation process, and evaluate if SSA has the capacity to effectively complete the implementation process.

In order to assess SSA’s progress to date in the implementation process and evaluate if SSA has the capacity to effectively complete the implementation, we obtained an understanding of the underlying process and procedures surrounding the implementation of the measure through interviews and meetings with the appropriate SSA personnel. In addition, we reviewed and evaluated the following:

• "Organizational Culture Project: Final Report," May 2000.

• Memo entitled "Organizational Culture Workgroup Recommendations,"

October 18, 2000.

• SSA Performance Plan for FY 2000.

• SSA Performance Plan for FY 2001, Revised Final Performance Plan for FY 2000.

• SSA 2000-2005 Strategic Plan "Mastering the Challenge."

We reviewed and evaluated the following documents to gain an understanding of the processes used to create the workforce plan and the pilot employee survey.

• "Workforce Planning at the Social Security Administration," February 2000.

• Memo entitled " Future Workforce Transition Plan—ACTION," June 5, 2000.
• Memo entitled "Quarterly Status Report on SSA’s Future Workforce Plan—INFORMATION," November 28, 2000.

• SSA Performance Plan for FY 2000.

• SSA Performance Plan for FY 2001, Revised Final Performance Plan for FY 2000.

• SSA 2000-2005 Strategic Plan "Mastering the Challenge."

• SSA Employee Survey RFQ, released August 2000.

• Pilot Employee survey.

As part of this engagement, we evaluated the appropriateness of each of the performance measures with respect to GPRA compliance and SSA’s APP. To do so, we determined whether the specific indicators and goals corresponded to the strategic goals identified in SSA’s APP, determined whether each of these indicators accurately measure performance, and determined their compliance with GPRA requirements.

APP Annual Performance Plan
ASP Agency Strategic Plan
FY Fiscal Year
GPRA Government Performance and Results Act
OCW Organizational Culture’s Workgroup
OHR Office of Human Resources
OWA Office of Workforce Analysis
PM Performance Measure
PwC PricewaterhouseCoopers LLP
SSA Social Security Administration