<DOC> [109th Congress House Hearings] [From the U.S. Government Printing Office via GPO Access] [DOCID: f:30381.wais] OVERSIGHT HEARING ON THE TRAINING PROVIDED TO VBA CLAIMS ADJUDICATORS AND THE STANDARDS USED TO MEASURE THEIR PROFICIENCY AND PERFORMANCE ======================================================================= HEARING BEFORE THE COMMITTEE ON VETERANS' AFFAIRS HOUSE OF REPRESENTATIVES SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS ONE HUNDRED NINTH CONGRESS SECOND SESSION _____________ SEPTEMBER 13, 2006 _____________ Printed for the use of the Committee on Veterans' Affairs Serial No. 109-62 _____________ U.S. GOVERNMENT PRINTING OFFICE 30-381 PDF WASHINGTON : 2007 --------------------------------------------------------------------- For sale by the Superintendent of Documents, U.S. Government Printing Office Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; DC area (202) 512-1800 Fax: (202)512-2250 Mail: Stop SSOP, Washington, DC 20402-0001 COMMITTEE ON VETERANS' AFFAIRS STEVE BUYER, Indiana, Chairman MICHAEL BILIRAKIS, Florida LANE EVANS, Illinois, Ranking TERRY EVERETT, Alabama BOB FILNER, California CLIFF STEARNS, Florida LUIS V. GUTIERREZ, Illinois DAN BURTON, Indiana CORRINE BROWN, Florida JERRY MORAN, KANSAS VIC SNYDER, Arkansas RICHARD H. BAKER, Louisiana MICHAEL H. MICHAUD, Maine HENRY E. BROWN, Jr., South Carolina STEPHANIE HERSETH, South JEFF MILLER, Florida Dakota JOHN BOOZMAN, Arkansas TED STRICKLAND, Ohio JEB BRADLEY, New Hampshire DARLENE HOOLEY, Oregon GINNY BROWN-WAITE, Florida SILVESTRE REYES, Texas MICHAEL R. TURNER, Ohio SHELLEY BERKLEY, Nevada JOHN CAMPBELL, California TOM UDALL, New Mexico BRIAN P. BILBRY, California JOHN T. SALAZAR, Colorado JAMES M. LARIVIERE, Staff Director SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS JEFF MILLER, Florida, Chairman SHELLEY BERKLEY, Nevada, JERRY MORAN, Kansas Ranking JEB BRADLEY, New Hampshire, Vice Chairman TOM UDALL, New Mexico GINNY BROWN-WAITE, Florida LANE EVANS, Illlinois PAIGE MCMANUS, Subcommittee Staff Director C O N T E N T S September 13, 2006 Page Oversight Hearing on the Training Provided to VBA Claims Adjudicators and the Standards Used to Measure their Profi- ciency and Performance .................................... 1 OPENING STATEMENTS Chairman Miller ............................................ 1 Hon. Shelley Berkley, Ranking Democratic Member ............ 2 Prepared statement of Ms. Berkley .......................... 22 STATEMENTS FOR THE RECORD Hon. Ginney Brown-Waite .................................... 23 WITNESSES Walcoff, Michael, Associate Deputy Under Secretary for Field Operations, Veterans Benefits Administration, U.S. Depart- ment of Veterans Affairs .................................. 3 Prepared statement of Mr. Walcoff .......................... Wise, Lauress L., Ph.D., President and Chief Executive Officer, Human Resources Research Organization ..................... 11 Prepared statement of Dr. Wise ............................. 36 Cox, J. David, R.N., National Secretary-Treasurer, American Federation of Government Employees, AFL-CIO ............... 13 Prepared statement of Mr. Cox .............................. 45 Smithson, Steve, Deputy Director for Claims Services, Veter- ans Affairs and Rehabilitation Division American Legion ... 15 Prepared statement of Mr. Smithson ......................... 56 OVERSIGHT HEARING ON THE TRAINING PROVIDED TO VBA CLAIMS ADJUDICATORS AND THE STANDARDS USED TO MEASURE THEIR PROFICIENCY AND PERFORMANCE _______________ Wednesday, September 13, 2006 U.S. House of Representatives, Subcommittee on Disability Assistance and Memorial Affairs, Committee on Veterans' Affairs, Washington, D.C. The Subcommittee met, pursuant to call, at 2:30 p.m., in Room 334, Cannon House Office Building, Hon. Jeff Miller [Chairman of the Subcommittee] presiding. Present: Representatives Miller, Berkley. Staff Present: Paige McManus, Majority Counsel and Mary Ellen McCarthy, Minority Counsel. Mr. Miller. Good afternoon, everybody. Our Ranking Member is here, so we will bring this meeting to order. As we all know, the number of veterans filing claims for compensation has increased every year since 2000, and the claims that are being filed are becoming much more complex. As we are all very well aware, many veterans are upset with the time that it takes to adjudicate a claim for compensation benefits. As of July, the most recent data available, it is taking the Veterans Benefits Administration an average of 174 days to adjudicate a claim, with an accuracy rate of 88 percent. The ability to provide timely and accurate benefits is dependent not solely on increasing staffing levels, but providing proper and thorough training. In recognition of this, the Committee recommended an additional 400,000 above the administration's request in its fiscal year 2007 views and estimates to fund training and certification initiatives. Congress and certainly VA must ensure that current and new employees have the skills and knowledge necessary to render accurate and fair decisions the first time. VBA has a number of programs and tools that it utilizes to train new Veteran Services Representatives, and to retrain or refresh more senior VSRs and Rating Veteran Service Representatives. The VBA has implemented a skills certification test to assess the knowledge base of claims adjudicators, and to provide additional training when necessary. However, it is my understanding that a great many employees either don't have the time needed to devote to training, or are failing basic competency tests. We meet today to receive testimony on the types of training provided to claims examiners, the standards that are being used to measure proficiency and performance, and just what VBA is doing to enhance performance. I want to take this opportunity to recognize our Ranking Member, Ms. Berkley, for comments. Ms. Berkley. I thank you very much, Mr. Chairman, for holding this hearing, and welcome to all of you. VA employees who decide critical benefits issues for our nation's veterans must obviously have proper training in order to fulfill the VA's mission. I am very pleased that we are giving attention to this matter today. I am concerned that the majority of experienced VSRs who have taken open-book, open-resource tests were not successful in passing these tests. I am also concerned that files reviewed by Committee staff, and claims brought to the attention of the Committee, often contain obvious errors, which VA managers quickly agree to correct. According to Mr. Walcoff's testimony, the VA provides national comprehensive training to regional office staff. However, both the employees union, American Federation of Government Employees, and the American Legion, report wide regional variations in training. Having a national plan does not guarantee that a national plan will be effectively implemented. The union testimony concerning VBA excluding labor representatives from meetings about training and performance standards is, quite frankly, very disturbing. Labor and management need to work together to improve the training provided to VA employees. Veterans who have honorably served our nation deserve quality decisions in a timely manner in order to receive the benefits and services they deserve. I am anxious to hear your testimony, both panels, and I thank all of you for being here today, and look forward to hearing what you have to say. Mr. Miller. Thank you very much, Ms. Berkley. Our first panel is already seated at the table. Other members will be coming and going as we have other hearings going on this afternoon. Testifying on behalf of the Veterans Benefits Administration is Mr. Michael Walcoff, Associate Deputy Under Secretary for Field Operations. He is accompanied by Mr. James Whitson, who is Director of the Eastern Area. Ms. Dorothy MacKay is here today, she is the Director of the Office of Employee Development and Training, along with Ms. Janice Jacobs, Deputy Director of Policy and Procedures at the Compensation and Pension Service. Mr. Walcoff, you are recognized. Mr. Walcoff. Thank you, Mr. Chairman, and members of the Subcommittee. I appreciate the -- Mr. Miller. If you could pull that mic just a little bit closer. Mr. Walcoff. Sure. STATEMENT OF MICHAEL WALCOFF, ASSOCIATE DEPUTY UNDER SECRETARY FOR FIELD OPERATIONS, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS AFFAIRS; ACCOMPANIED BY JAMES A. WHITSON, DIRECTOR, EASTERN AREA VETERANS BENEFITS ADMINISTRATION; DOROTHY MACKAY, DIRECTOR, OFFICE OF EMPLOYEE DEVELOPMENT AND TRAINING, VETERANS BENEFITS ADMINISTRATION; JANICE JACOBS, DEPUTY DIRECTOR, POLICY AND PROCEDURES, COMPENSATION AND PENSION SERVICE Mr. Walcoff. I appreciate the opportunity to appear before you today to address training for Veterans Benefits Administration claims adjudicators, the standards we use to measure their proficiency and performance, and how we communicate changes in laws, regulations, policy, and procedures, to field station staff. I am being accompanied by my VBA colleagues; Mr. Jim Whitson, who is the director of our eastern area office and also the Project manager for C&P certification; Ms. Janice Jacobs, deputy director of compensation and pension service; and Dorothy MacKay, director of our office of employee development and training. Training is essential to every quality organization. VBA is committed to ensuring all employees have the opportunity to learn and develop essential knowledge, skills, and abilities. We recognize effective training as a core element of VBA's infrastructure, and key to our succession planning efforts, as we hire to replace the many experienced employees expected to retire over the next few years. VBA has deployed new training tools and programs to provide consistency in training. Newly hired veteran service representatives, or VSRs, and newly appointed rating VSRs participate in consistent, nationally-developed centralized training, followed by a national standardized training curriculum at their home offices. This fiscal year, the undersecretary for benefits, Admiral Daniel L. Cooper, mandated all claims examiners to have at least 70 hours of job-specific training. Most other employees must have at least 40 hours of training. In fiscal year 2007, the mandatory training for claims adjudicators will increase to 80 hours. VBA ensures regional offices have timely, accurate, and current information. C&P service issues guidance letters, called `` fast letters,'' to advise field employees on policy and procedural changes, and legal revisions. C&P service issued 23 fast letters in 2005, and 16 to date in 2006. Decision assessment documents, or DADs, explain the facts, reasoning, and holding of precedential court decisions, and the opinions of the office of General Counsel, and explain any impact on VBA. Notification of fast letters, DADs, and other changes, are e-mailed to field employees and posted on the C&P service's website. C&P service uses satellite broadcasts to give in-depth analysis on VBA procedures and policies by subject matter experts, and hosts monthly conference calls to discuss current hot topics and give instant feedback on questions and concerns. Regarding VBA, VSR, and RVSR national performance standards, a VBA organizational cornerstone to improve benefit delivery and enhance accountability is our system of individual performance assessment. All VSRs and RVSRs are subject to national performance standards that focus on key elements of quality, productivity, customer service, and timeliness. Local accuracy reviews are conducted using national quality review criteria. Agreements were reached with the American Federation of Government Employees prior to implementation. The current VSR performance standards have been in effect since October of 2005. The RVSR performance standards have been in effect since November of 2001. The VBA work group is reviewing these standards for possible revision. Regarding certification testing; in conjunction with our national performance standards, VBA developed a skill certification testing process to assess job proficiency. A memorandum of understanding was signed in January 2000 with AFGE for a certification program for VSRs, RVSRs and decision review officer positions. In December 2000, VBA contracted with the research group, the Human Resources Research Organization, or HUMRO, to help develop a certification instrument with VBA subject matter experts to assess knowledge and readiness of VSR GS 10 incumbents for promotion to the GS 11 level. A 100 question multiple choice open-book test is administered in morning and afternoon sessions. In August of 2003, 25 percent of the 298 participants passed the first validation test. In April of 2004, 29 percent of the 650 participants passed the test. After the first two tests, a mandatory 20-hour VSR readiness training curriculum was implemented. On May 3rd, 2006, 934 VSRs took the third test. During the afternoon portion, multiple test items were duplicated from the morning portion. The correct version of the test was given on June 7th, and the third test yielded an improved pass rate of 42 percent. To date, VBA has promoted 633 VSRs to the full-performance GS 11 level, through certification testing. We are expanding certification testing next, to the RVSR position, from the GS 11 level to the GS 12 journeyman level. Mr. Chairman, this concludes my testimony. I greatly appreciate being here today, and look forward to answering your questions. [The statement of Michael Walcoff appears on p. ] **********INSERT********** Mr. Miller. Thank you very much for your testimony. I think we both have some questions, and one of the issues I think that we probably both want to know about is when a case is reprimanded from the Board for further development, it is generally sent to the Appeals Management Center in Washington, DC. Is the employee who made the initial decision - first question - is the employee ever notified of what the deficiencies were, or what may have happened in the appeal? You know, I would think that if they are not, then it probably would help the R.O. directors develop a better understanding of how ratings happen, and where the issues are. Can you go into that just a little bit? Mr. Walcoff. Sure. Let me first say that when a case is remanded from the board, it is very often three, sometimes four years after it was originally worked, for a lot of different reasons. Very often, the case that is remanded back to VBA doesn't look like it did when it left the regional office. Additional information has been sent in by the veteran. Many, many cases are requiring a new exam, and that is a reason why a case might be remanded. Sometimes, the rating schedule may change. There might be a change in regulation that might cause for the case to be remanded. And certainly, in a small percentage of cases, a percentage of cases, it is similar to when the VSR finished it, or the rating specialist finished it, and it is because of an error that was made at that time. But it is a combination of all those factors that go into the remand rate. When we originally set up the AMC, our discussions with the prior secretary dealt with how many of the remands that came out of the board should be done at the AMC, and how many should be returned to regional offices. It was Secretary Principi's opinion that he wanted all the remands to be processed at the AMC. He felt that this was the most efficient way to work through the remands. He was concerned about the fact that there were a lot of older remands that were still out there, and he felt that this would be the best way to deal with that. We had some very serious discussions with him about why we felt that, at minimum, a certain percentage of these cases needed to go back to regional offices. One of the reasons, quite frankly, was that in people working these cases, and they have the sense that `` no matter what I put on this piece of paper, I will never see the case again,'' that there is almost a little bit of a disincentive in terms of being much more careful about what they are sending to the board, and making sure that it is ready for the case to be adjudicated by BVA. And we were concerned about that. We wanted to make it so that some of these cases do go back to regional offices. But more importantly, we felt that it was important that some of these cases go back to R.O.'s for training purposes. It is very important for people who are working on certifying cases to the board, that they get to see what types of cases are being remanded, so they can use that information in doing the work that they are currently doing. So it doesn't necessary to go back to the original person that prepared the case, because as I say, it is many years later. But they do see a good number of the cases. And they particularly see what we call the egregious cases, the ones where there is really absolutely no excuse for why this case ever left the regional office. And we want to make sure that those cases are reviewed by the regional offices, and by the management at those offices to make sure that it doesn't happen again. Mr. Miller. In its 2005 report on state variances in compensation payments, the VA Inspector General surveyed about a thousand employees on factors that affect consistency of rating decisions. The results showed that while 60 percent of the employees had positive opinions about the quality of the training that they were receiving, many feel that the training is not a high priority, and some 37 percent or so indicated that formal classroom training occurs once a quarter, or less often than that. Can you talk to us a little bit about what is being done? Mr. Walcoff. I would say that the Under Secretary, Admiral Cooper, has I think probably stressed training, training and consistency, probably more than any other subject since I have been working with him. I have been in Washington since 1998, and I have served under several under secretaries, and was here previously, back in the early '90s with two other under secretaries. I have never worked for anybody that was, I believe, as serious about training as Admiral Cooper is, and I think it is because of his background with all the years he spent in the Navy. I will tell you that it is one thing to say that training is important, and `` here is a mandatory requirement that you do 70 hours.'' But the real question is, how do you ensure that is actually happening? And I will tell you a couple different ways that I think we are going about doing that. One is I think we are providing very close oversight when we go to regional offices, to make sure that training is being done and is being done effectively. The C&P service review goes to about 18 offices a year, and to review the C&P operations at each office. And one of the main things that they look at is the training that is being done; whether the curriculum that is specified by the C&P service is being followed, and whether all employees are going to the training. In addition to that, our area directors visit their offices two, three, four times a year. And one of the things that they look at is the same thing: how is the training being done? And they will not just talk to the managers. They will actually go and talk to the VSRs, and talk to the rating specialists, and ask them, you know, `` How much training have you received,'' you know, `` Did you actually attend this class? Were you at the satellite broadcast?'' Those types of things, to make sure that it is happening. Secondly, we have a log that has been put together by Dorothy's staff that requires the managers to report every hour of training that is given to employees, so that we can verify that 70 hours of training this year, in this case, was actually given to every employee in regional offices in the service center. So we also review the logs. And the third thing that we do, and this is something that I haven't seen in all the years I have been working, is that Admiral Cooper personally reviews the training report that is done twice a year by each regional office, that goes into great detail about all the training that they have done during that previous six months. And this is not something that comes in and he has Dorothy give him a summary report. He actually has to see the reports. He reads every one of them, marks them all up with comments, and sends them out back to her and to me, you know, with questions, and wanting us to follow up on things, obviously showing that he is sincerely interested in it. So I think that aside from having the requirement, there is a lot of follow-up to make sure that it is actually being done. This is a new requirement. I mean, this is the first year that we actually had that mandatory 70 hours. And I think it was a little bit of a shock to our organization, frankly, that somebody did more than just say, `` Yeah, I think training is important,'' but actually put out a mandatory requirement, you know, saying that `` You will do training.'' That has had an impact on the organization. Mr. Miller. Ms. Berkley? Ms. Berkley. Thank you, Mr. Chairman. In reviewing panel two's testimony, a couple of things struck me, and I would like to ask you questions based on the testimony that was submitted by the other panel. According to AFGE's written testimony, it indicated that unrealistic performance standards and inconsistent training adversely impact the quality of employees' work. They report also being excluded from the task force meetings where current training and performance standards are discussed. Can you share with us why we would exclude these key stakeholders, and why they are not at the table when discussions of this magnitude are being -- when they are discussing these issues? Mr. Walcoff. Well, first of all, I would tell you that I think we have a very positive relationship with AFGE. The speaker that will be representing AFGE here in the second panel is somebody that I have worked personally with for many years, and I think that while we may disagree on individual issues, I think he would agree that the VBA and AFGE have had a good relationship. I think that we certainly accept that there is an appropriate role for labor, for AFGE. We certainly understand all the things that we are responsible to do in terms of our relationship with them, and I think that we very clearly meet every requirement that we have in terms of our relationship. We do include them on many of these types of issues that we work on. The RVSR performance standards, for instance, that are in place right now, the union had a representative on that team. They participated in setting up those standards. That is aside from the fact that we then negotiated an agreement with them about the standards that are in question right now. The VSR standards, which is I think what David is referring to in his statement, we negotiated an agreement with them on the implementation of those standards. And the disagreement that we have right now with them that he has mentioned in his testimony has to do with a requirement in the MOU for us to review the actual performance under those standards, and make changes, and there is a difference of opinion as to whether we have done that or not. But we certainly have worked with the union on the implementation of those standards, and met every requirement that is there for our labor-management relationship. In terms of the standards being too difficult, I respectfully disagree. I think that some of our managers feel that we should have set the floors higher. We make it clear that it is not just a productivity requirement; that there is an absolute quality requirements in there, as well as a timeliness and a customer-service requirement. They are all considered critical. And if you look at the performance of the field in those standards, right now we have a pass rate of over -- in terms of meeting all the requirements of their position -- of over 80 percent within the VSR position. And that goes up every month, as people become more familiar with the changes that we made almost a year ago in the way we measure those standards. Some of the problems that we have had had to do with the tracking system that is designed to go with those standards, called Aspen. There has been -- it is a new system, and it is taking quite a while for some of our employees, as well as managers, to understand how to use it. And as we reviewed the pass rates in preparation for the grievance that is mentioned in David's testimony, we found that there were some R.O.'s that weren't using the tracking system correctly, and that was why some of the pass rates look lower than they actually are. But we are very confident that the pass rate is over 80 percent, and going up every month. Ms. Berkley. Do you think there is inconsistent training? And does it adversely impact on the quality of the employees' work? Mr. Walcoff. You know, inconsistent training is something that we are always very concerned about, and we have done everything, I think, that we can to try to deal with that. That is why we have not only -- not only do we have a national curriculum, starting with the basic challenge training for all of our new employees, where they go to a centralized place and spend three weeks there, and then when they go back to their regional offices, spend 23 weeks taking a national TPSS curriculum that everybody is required to take. But for journeyman VSRs and rating specialists, we get the 70-hour requirement, where C&P service has listed a number of courses that are mandatory for every individual during that year, and also specifies `` This is the way you will teach it. Here is a satellite broadcast,'' you know, `` Here is a video,'' that type of a thing. So certainly, mechanisms are out there to ensure that there is consistent training. We track the logs to make sure that everybody is doing what they are supposed to do consistently. You know, in an individual -- Ms. Berkley. There is adequate oversight, in your opinion? Mr. Walcoff. I believe there is. I believe there is, ma'am. Ms. Berkley. Let me draw your attention to the American Legion testimony. In it, they have indicated that the focus on end-product production quotas in regional offices leads to poor-quality decisions and appeals. When and how did the VA last evaluate the usefulness of end-product code measure of performance? And is there perhaps a better method of measurement? Mr. Walcoff. I think that everybody, or just about everybody, would agree that we would like to evolve away from the basic end-product system, to more of an issues-oriented system, and I have been to hearings where this has been discussed. And I think you know that up until about a year ago, we really didn't have the ability to accurately measure the work that we are doing or receiving on an issues-based basis. With RBA 2000 now being mandatory, and everybody using it, that is really a vehicle that really does allow us to be able to count issues versus cases. And that is really the first big step toward moving away from an end-product system. It is certainly a more accurate way to measure quality. It is a more accurate way to measure our production. And it would do a lot in terms of some of the concerns even that AFGE has about how we measure how productive, you know, an individual is. We have talked a lot about it. There are some initial steps taking place for us to begin doing that. It is going to be a process. I mean, it is certainly something that were going to have to work with, with all of you all on, in terms of explaining what we are doing, why we are doing it, and how that will affect our performance, because we want to be able to give you apples and apples to be able to compare in terms of whether we are improving or not. Certainly we would work with service organizations, and with the union, as we migrate from the one system to another. It is not something that is going to be done over a period of months. But we have had discussions with Admiral Cooper, and I know that he is committed that this is something that were going to want to do. Ms. Berkley. Is there anything that we can do to help implement that, in getting away from the end-product system? Because I agree with you; it won't take months. But how do we, as members of this Committee, enable you to do this and not be here 20 years from now talking about the same thing? Mr. Walcoff. One of the things that I think will be helpful is, you know, any time we make a major change in the way we measure something, you know, I think that there is always a concern -- and as I say, I have been around in headquarters for a while, and I know that sometimes there is a concern, `` Well, they are kind of changing the name of the game a little bit to maybe just take away attention from what actual performance is, and that sort of thing.'' And I think certainly that is a legitimate concern that our stakeholders certainly should be thinking about. What I would want to do in terms of working with you is to try to sit down and talk about, `` Okay, what is it that you would need for reassurance as we evolve from one system to another?'' So that you could see not only that we are moving, you know, in a direction that we both think we need to, but you can also monitor on us in your oversight capacity as to how we are performing, and be able to do it on some consistent plane, rather than in two different languages, sort of, you know, so to speak. And that is something we will be able to hopefully work with you on, and you know, I think we can come to some agreement on that. Ms. Berkley. Okay. Thank you, Mr. Walcoff. Mr. Miller. Thank you very much. We appreciate your testimony. Thank you for those that joined you today. If you would, if you could have a member, you or a member of your staff, if they could remain behind we would appreciate it, for the next panel. Thank you very much. Mr. Walcoff. Okay, thank you. Mr. Miller. And if the second panel will make their way forward. While you are getting settled I am going to go ahead and introduce everybody if it is okay. Dr. Lauress Wise is president of the Human Resources Research Organization, the group VBA contracted with to develop the skills certification testing program. Mr. Jeffrey David Cox is the national secretary-treasurer of the American Federation of Government Employees. And Mr. Steve Smithson, he is with us, is the deputy director for claims services at the Veterans Affairs and Rehabilitation Division of The American Legion. And each of you we would ask, if you would, hold your testimony to five minutes, since there are three of you. Your full testimony will be entered into the record. It will be printed in its entirety, as it was submitted. Dr. Wise, welcome, and you may begin. STATEMENTS OF LAURESS L. WISE, PH.D., PRESIDENT AND CHIEF EXECUTIVE OFFICER, HUMAN RESOURCES RESEARCH ORGANIZATION; J. DAVID COX, R.N., NATIONAL SECRETARY-TREASURER, AMERICAN FEDERATION OF GOVERNMENT EMPLOYEES, AFL-CIO; STEVE SMITHSON, DEPUTY DIRECTOR FOR CLAIMS SERVICES, VETERANS AFFAIRS AND REHABILITATION DIVISION, AMERICAN LEGION STATEMENTS OF DR. LAURESS WISE Mr. Wise. Thank you. Good afternoon, Mr. Chairman. I am Lauress Wise, president of the Human Resources Research Organization, known less formally as HUMRO. HUMRO is a nonprofit 501(c)3 research and development organization established in 1951 that worked with government agencies and other organizations to improve their effectiveness in developing and managing their human resources. I have been asked to testify today about the work that HUMRO has done for the Veterans Benefits Administration on their program for certifying essential skills for veteran service representatives. For brevity, veteran service representatives are referred to as VSRs. These service representatives play a key role in seeing that our veterans receive a full array of benefits to which they are entitled. Performance at the highest level of this position requires a thorough understanding of an extensive set of policies and procedures, and skill in identifying appropriate applications of these procedures to individual circumstances. The skill certification program embarked on by the VBA is critical to ensuring that service representatives have the knowledge and abilities needed to perform their jobs effectively. My written testimony, which I ask be entered into the record, describes in some detail HUMRO's efforts to develop the VSR skill certification test. I will just cover briefly the essential points here. Our work began in 2001 with an analysis of critical tasks performed by the VSRs and the knowledge and skills required to perform these tasks effectively. HUMRO worked with a design team that included VBA management, expert job incumbents, union representatives, and members of the veteran service organizations, to review the job analysis results, and develop a blueprint for the certification test. The blueprint identified content areas to be covered by the test and specified the number of test questions to be included for each of these content areas. We worked with the design team to develop, review, and provide high-quality test questions. A pilot test was conducted to screen individual questions followed by a field test of the entire certification process. Two different approaches were taken to accessing the validity of the resulting test scores for certifying a candidate's qualification for the GS 11 position. In a content validity study, experts affirmed that each of the test questions measured knowledge and skills essential for job performance. We also conducted a criterion-related validity study. In that study, test scores were found to be significantly correlated with job performance ratings made by each examinee's supervisors. The test development and validation process conformed fully to professional standards for test development and use. The resulting certification test is a valid and important tool for ensuring essential skills for those promoted to the highest level VSR job. I was also asked to comment on the issues surrounding the May 3rd administration of the skill certification test. Two forms of the test ere administered to 934 candidates. The reason for administering two forms was so that examinees sitting next to each other would not get the same questions in the same order. Each test form included 100 scored items and 20 additional questions being pilot tested for future use. The two forms had 67 operational, or scored, items in common, albeit in different locations within each test form. Because of the length of the test, the test is split across two sessions; one in the morning and one in the afternoon. Candidates received a different test booklet for each of these two sessions. Due to a processing error at HUMRO, some of the questions for the afternoon booklets were inserted into the incorrect test form. This error resulted in duplicating some questions from the morning session in the afternoon session booklets for that corresponding test form. Quality control procedures in effect at the time included a review of each test but did not include a thorough comparison of the morning and afternoon booklets within a test form. Consequently, this processing error was not caught prior to the test administration. The week following the May administration, HUMRO Vice President Beverly Dugan, our project director, Dr. Patricia Keenan, and I met with VBA leadership to discuss the problem and identify methods for providing valid scores to the May administration examinees. Our discussion identified several possibilities, including using some of the pilot test items to construct an 80-item test; ignoring the redundancy and scoring each of the duplicate items to provide a 100-item test; and conducting a supplemental administration, using the items that were originally intended to be included in each of the afternoon booklets. The solution selected by the VBA management was to conduct a supplemental test, and administer the items that were originally intended to be presented in each of the afternoon tests. This allowed everyone to be scored on 100 separate items, to keep the test mapped to the blueprint exactly as designed, and made the May 2006 administration much more equivalent to the operational field test and validity test, and to those administrations planned for the future. The supplemental test administration was held on June 7th, 2006. After both portions of the test were scored, 42 percent of those who took the test passed. The supplemental testing did create an inconvenience to examinees, and additional burden to those who administered the test, but the end result was an assessment that covered the content framework as intended, with questions and scores that were psychometrically sound. HUMRO's staff have reviewed the factors that led to the error in assembling the test booklets. One such factor was the limited time available for assembling and checking booklets. The VSR job continues to evolve. New types of cases are often added to the caseload. Newly electronic tools and databases are developed, and more pension cases are being moved to pension maintenance centers. A workshop to review test questions, to be sure that they reflected current policy and processes, was held in April of 2006. Item writers reviewed all of the items in the bank, revised many of them, and updated the references. The revisions were more extensive than originally anticipated, and the work to update the item bank was completed late the week prior. We had only two days to select the forms. In retrospect, it was clear that more time is needed. We have now implemented procedures to prevent future occurrence of these problems, both by allowing more time, and incorporating more thorough checks of the test booklets. In summary, the VSR test is an important tool for improving the effectiveness of the VSR workshop, and serving the benefit needs of our veterans. Mr. Chairman, that concludes my testimony. Thank you for the opportunity to appear today. [The statement of Lauress Wise appears on p. ] **********INSERT********** Mr. Miller. Doctor, thank you very much for your testimony. What I would like to do if we could before we go to questions is let the other panelists go ahead with their remarks. Mr. Cox, you are recognized. STATEMENT OF J. DAVID COX Mr. Cox. Thank you, Mr. Chairman. Chairman and members of the Subcommittee, good afternoon. I am J. David Cox, national secretary-treasurer of the American Federation of Government Employees. Thank you for inviting AFGE to testify today. I ask that my written statement be submitted for the record. Mr. Miller. Without objection. Mr. Cox. As the purse Vice President of AFGE's national VA council and a local president, I have visited a large number of VBA regional offices. What I observed from the front lines is that VSRs and rating specialists are under tremendous pressure to process enough cases to meet performance standards. They stay late into the evening and work weekends. They study on their own time because training is often cut short to make the numbers. Despite all this, many of them cannot make the numbers. They are further demoralized by a confusing certification test. VSRs and rating specialists want to help veterans get an accurate and prompt decision on their claims. Many are themselves service-connected veterans. VBA training is clearly an investment worth making. AFGE believes that genuine collaboration on training and performance measures will make this critical investment go far. Lawmakers, managers, employee representatives, and veterans groups, have to work together to come up with training and standards that are workable, doable, and durable. Unfortunately, VBA seems less and less open to collaboration than in the past. After we have worked together to develop the CPI model, we have now been left out of key task force meetings. After we worked together on national performance standards, they have refused to address problems that have been developed. Even though we developed the VSR certification by agreement, VBA is refusing to work with us to fix what isn't working. The role of on-the-job training for VSRs and rating specialists cannot be underestimated. By VBA's own estimates, it takes at least two to three years for a new VSR to be fully productive. Yet, what VBA is doing about an impending shortage of experienced adjudicators due to retirement and high turnover among frustrated new employees. AFGE offers the following recommendations: a joint VBA stakeholder team should develop a national mandatory training plan, with strong oversight. We need a uniform curriculum, training schedule, and a set of best practices. The oversight process should be transparent to all stakeholders, with regular reports to Congress to make sure that VSRs and rating specialists everywhere get the same quality and amount of training, and rotate through all adjudication teams. VBA should develop a cadre of trainers skilled in both teaching and veterans benefits. Currently, trainers are selected without consistent criteria. All trainers should be of the caliber one is likely to find at the VBA Academy. Continuing education should be mandatory and nationally uniform. Supervisors should be required to hold weekly meetings to go over new laws, regulations, court cases, and best practices. Current performance standards need revision. One third of VSRs and rating specialists are failing to meet unrealistic production quotas. CPI specialization should also be factored in. Training for skills and certification tests should be nationally uniform. The first two validity tests for VSR certifications had a very low passage rate, and more than half failed the May 2006 test. I can't help but wonder whether any managers have ever had to take this test, and what their pass rate would be. The test is supposed to be a culmination of the training, but there is a real disconnect between the two. Trainers do not even know what to cover. Employees are not told how many questions they need to get right, which questions they got wrong, and what the correct answer is. Something is wrong when two employees with identical scores test at different times, and only one passes. Finally, we hope the Subcommittee will consider the related issue of classification of the VSR and rating specialist positions. Unlike comparable adjudication jobs at other federal agencies, the VSR career ladder ends at the GS 10. We currently understand that VBA recently completed a review of these classifications. Once again, we would like to be part of this process. We look forward to working with Chairman Miller and Ranking Member Berkeley, to ensure that VBA claimants get the best services possible. Our veterans deserve nothing less. Thank you, Mr. Chairman, for allowing me to be here today. [The statement of J. David Cox appears on p. ] **********INSERT********** Mr. Miller. Thank you, Mr. Cox. Mr. Smithson, you are recognized. STATEMENT OF STEVE SMITHSON Mr. Smithson. Mr. Chairman and members of the Subcommittee, thank you for the opportunity to present the American Legion's views on the training provided to Veterans Benefits Administration claims adjudicators and the standards used to measure their proficiency and performance. We commend the Subcommittee for holding this hearing to discuss these important issues. I am going to limit my remarks to emphasize major points discussed in my written testimony, and ask that my written statement be entered into the record. Mr. Miller. Without objection. Mr. Smithson. The American Legion appreciates the importance the Undersecretary for Benefits has placed on training, and the improvements that have taken place, including centralized training for regional office staff conducted by the compensation and pension service. However, in our opinion, the culture of VA's claims adjudication has been and continues to be production-driven. This results in everything else, including quality assurance and training, taking a back seat. If this culture is ever going to change, VA, in addition to the centralized training currently taking place, must consistently implement national training requirement or standard in each regional office for all employees, new hires and experienced alike. It must also be implemented with strong oversight directly from the Under Secretary for Benefits' office, and it must have teeth. Management stations not in compliance must be held accountable, or the effort will not be successful. Additionally, there must be an emphasis placed on using information obtained from BVA decisions, DRO decisions, and errors noted in the National Star reviews, and other internal quality reviews. Such data should be tracked, examined for patterns, and used in formal, customized training at each R.O. Although such data is being collected and the American Legion's quality review visits at VA regional offices have identified some stations that have been conducting such training, it needs to be done across the board at each R.O. Unless regional office staff, both managers and individual adjudicators, learn from their mistakes and take corrective action, there will continue to be a high rate of improperly adjudicated claims, resulting in consistently high appeals rate, and a subsequent high VBA remand and reversal rate. Performance standards, which tend to go hand-in-hand with the production-first mentality entrenched in VBA is another topic that needs to be addressed. Performance standards of claim developers and raters are centered on productivity as measured by work credits, known as end products. Both veteran service representatives and rating service representatives have minimum national productivity requirements that must be met each day. Unfortunately, the end product work measurement system, as managed by the VA, does not encourage regional office managers to ensure that adjudicators do the right thing for the veteran the first time. For example, denying a claim three or four times in the course of a year before granting benefits sought allows for several end-product work credits to be counted just for this one case, rather than promptly granting the benefit and taking only one work credit. In the view of the American Legion, the need for substantial change in VBA's work measurement system is long overdue. A more accurate, reliable work measurement system would help to ensure better service to veterans. Ultimately, this would require the establishment of a work measurement system that does not allow work credit to be taken until the decision in the claim becomes final; meaning that no further action is permitted by statute, whether because the claimant has filed to initiate a timely appeal, or because BVA rendered a final decision. Lastly, we are pleased that C&P has begun implementing a job certification test for VSRs, and we look forward to similar testing being established for RVSRs and DROs. However, we are concerned that the current testing taking place is not required as a condition of employment for the position. although successful completion of the test is required for promotion or transfer to the rating board, it is still optional. The ultimate goal of a proficiency or competency testing should be to ensure that an individual in any given position is competent, proficient, and otherwise qualified to perform the duties required of that position. This goal will not be achieved if testing is not mandatory and remedial training or other corrective action is not required for those who do not successfully pass the test. Although this concept may not be embraced by some, the ultimate goal is to have qualified and competent staff who will provide the best possible service for our veterans. In closing, the American Legion realizes that VBA faces many difficult challenges during the upcoming fiscal year. Although we have offered our suggestions and comments, we realize that there is no easy solution, and we will continue to work closely with VA to ensure that our nation's veterans receive the benefits to which they are entitled. That concludes my testimony and I wills be happy to answer any questions. [The statement of Steve Smithson appears on p. ] **********INSERT********** Mr. Miller. Thank you very much, and I will start with you. You talked in your written testimony about too few experienced supervisors were available to provide trainees, I guess, with proper mentoring and quality assurance. Can you give me some type of an indication as to what you would think an experienced supervisor would be? Mr. Smithson. An experienced supervisor would be somebody that has been on the job for a number of years, has worked up through the system, and has proven themselves to be competent through their work, the performance standards, and other measures. Mr. Miller. Mr. Cox, you had talked about VA failing the VSRs by not adequately preparing them to take the skills certification test. Did I read that correctly? How much of the onus of passing a test falls on the employee? Mr. Cox. There would be a part that would fall on the employee, certainly, sir. However, much of the training material for the training for the test, and for the employee to study, is on the computer, sir. And I wouldn't even raise the issue of an employee taking a computer home at this point to study for the test. But I mean those type of things are the things readily available. And at the same token, this is not a job that an employee comes with a ready set of skills. I am a registered nurse by profession. I come to the VA with a set of skills required for that job. With a VSR, that person comes to the VA, and it is on-the-job training, and it is all done there on the job. Mr. Miller. Would AFGE support mandatory testing as a condition of employment? Mr. Cox. I think AFGE would have to look very, very closely, and be concerned with the mandatory testing as a condition of employment, sir. We don't believe that is done at other federal agencies with similar type jobs, similar to the claimant jobs; Social Security being one of them. Mr. Miller. Dr. Wise, you indicated that some of the test questions in the pilot test were dropped because participants indicated that there were problems with those questions. Can you give us an idea of what those might have been? Mr. Wise. Well, there certainly are instances where the rules may have evolved since the question was written, so that no longer clear which is the correct option on the question, would be an example. Mr. Miller. Ms. Berkley? Ms. Berkley. Thank you, Mr. Chairman. The State of Nevada has the fourth highest remand rate in the nation, and I have a lot of very unhappy veterans, and I have been trying for quite a while to figure out what we could do to make this work a little bit better. I have questions for each of you, and again, let me thank you all for being here and sharing your knowledge with us. Mr. Smithson, in your experience, do regional offices which provide consistent, high-quality training have fewer errors and lower remand rates? Mr. Smithson. Could you repeat the question please? Ms. Berkley. Yes, I can. Do regional offices - this seems like a no-brainer to me, but I am kind of curious, regarding your experience. Do regional offices that provide consistent, high-quality training have fewer errors and lower remand rates? Have we been able to track high-quality training, better service on the job, lower remand rates? Mr. Smithson. In our experience, there is more satisfaction. Veterans have more satisfaction from receiving decisions from those offices, yes. Ms. Berkley. I listened to your testimony, but I have to tell you I think you speak faster than I hear. So I would appreciate if perhaps you spoke a little slower. Do you have any specific recommendations for revision of VBA's workload management system? Mr. Smithson. Again, I think we need to -- are you talking about the end-product, the credit, work credit? Ms. Berkley. That could be part of it. Sure. Mr. Smithson. For the work credit, I think we need to look at not giving credit for a case until it becomes final, whether that be the one-year period expires and the veteran does not file a timely appeal, or the BVA renders a final decision. Ms. Berkley. When we talk about VBA's workload management system, is there a magic number that -- and I know it can't be the same number for every employee; everybody works differently. But is there a point at which our VBA employees are so overloaded that they can't possibly provide the quality service that our veterans need? Mr. Smithson. I think some stations, they are definitely overworked, not enough staff. When we do our quality review visits, not only do we look at the decisions for quality errors; we talk to the VBA staff, and a lot of them tell us that, you know, there is not enough people, the standards are too high, they are not realistic. Some stations, we encountered in our experience, in addition to the national standard have their own standards that may be above the production standards, that may be above what the national standard is, because of their backlog and their particular situation. Ms. Berkley. Okay. Mr. Cox, welcome. Are the problems that you described in your testimony with training widespread, or more problematic to specific regional offices? Is it just across the board? Mr. Cox. I think it is across the board. I think the training, that there is not consistency with the training. And also, dealing with adult learning is a creature unto itself, as I think we are all aware. Ms. Berkley. I would hate to have to go to law school again, that is for sure. Mr. Cox. Yes, I wouldn't want to go to nursing school again today, either. But I think people learn by different modalities. Some do well with online-type training. Others need a standard classroom, those type things. And so the training that meets the needs of the individual in how they learn to perform that job -- again, I think one very specific thing about these jobs in VBA are you don't come with a ready set of native skills for them. You have to get that training there at the job site. I mean, there are people in these VSR positions that come with a high school diploma, and some come with graduate degrees that do the same work, but they must learn all that work there. Ms. Berkley. But doesn't learning have to be somewhat standardized, the methods -- I think in our school systems across the country, there are kids that learn different ways, but I don't think that we can provide a specific type of learning path for one student and one for another and another. Is there a way that we could standardize this nationally, and still be able to provide the training that our VBA employees need? Mr. Cox. I think that you could get the standardized training. I would look at what is done at the VBA Academy in Baltimore. There are people -- that is their job, to do nothing but training. It is not a collateral-type duty or things that they do. They are trainers, and they do that work consistently over and over. And I do applaud the VBA. I think the training academy in Baltimore has done an excellent job, and will continue to do an excellent job, and that there are good lessons to be learned from the things that are housed there. Ms. Berkley. Let me ask you a couple of questions -- part of your testimony concerned me because I am a strong union supporter, and a great believer that unions and management working together certainly provide a much better product. Can you describe the efforts the union has made to engage VBA in collaboration on training and performance issues, and has there been a change in behavior? And if so, to what do you attribute that change of behavior to? Mr. Cox. I would think, you know, also Mr. Walcoff responded earlier, and I have always had a great working relationship with Mr. Walcoff and his colleagues at VBA, as well as many of us at AFGE. However, and I would, you know, say that in his comments, it is `` We comply with the law, we comply with the contract, we do the things that meets our labor-management obligation.'' However, many times we have to do more. We have a veteran, your constituents, at the end of this process, waiting for their claims to be processed, and be processed correctly. So to just comply with the law, with what it says in the labor-management relationship, I don't believe is enough. There must be a true partnership, a collaborative relationship of working together, and meeting the needs, clearly identifying problems, and arriving at a solution, and sometimes sharing information that either side kind of holds private. Ms. Berkley. Thank you very much. Mr. Wise, how does the pass rate of 25 percent and 42 percent for the open-book VSR test compare to other similar occupational tests with an open-book format? That seems extraordinarily low, if you have got the answers at your fingertips. Mr. Wise. Well, I would agree. And actually, I am not familiar with that many instances of open-book tests, which I think is a fairly unique feature of this assessment, in allowing the examinees access to the resources, most all the resources they would normally have on the job. Ms. Berkley. And you know, I found that kind of interesting because when I was in college and law school, if I took an open book test, I always performed worse, because I think I was expecting that I would be able to get the answers really quickly during the test-taking time, and it was just a disaster. So I am not sure that is a good idea. Mr. Wise. Well, it is an important feature of the assessment, that in order to ensure its validity we try and replicate the job as closely as possible. So for that reason, open book does seem appropriate to this exam. Ms. Berkley. Does the low pass-rate suggest that a number of experienced employees do not have the requisite knowledge, skills, and aptitudes to perform their current jobs adequately? And if that is not the case, to what do you attribute the low test scores? Mr. Wise. The VBA management and, you know, and the experts that reviewed it set the pass scores at a level that was adjudged to be the point at which people that were just sufficiently qualified would need to score, in order to demonstrate the kind of skill that is required for this higher-level position. The lower passing rates would indicate that many of the candidates who took the exam don't yet have those skills, but they do have the opportunity to go back and study and take the exam again. Ms. Berkley. Okay, thank you very much. Mr. Miller. Thank you very much, Ms. Berkley. That is all the questions that I have, too. And everybody's testimony has shown that the training claims examiners receive is vitally important to achieving VBA's mission. As I said in my opening statement earlier, the adjudication of claims is a complex and cumbersome process, and I certainly appreciate the importance that Admiral Cooper has placed on training by mandating the 80 hours of training annually, and hope that the regional offices' directors are taking heed of this mandate. I look forward to working with VBA and its stakeholders to ensure that proper training and skills development is at the forefront of claims development. I appreciate everybody's time for coming and attending today. Without objection, members will have five legislative days to submit materials for the record, as well as post-hearing questions to the witnesses. With nothing further, this hearing is adjourned. [Whereupon, at 3:29 p.m. the hearing was adjourned.]