|
BACKGROUND/RATIONALE:
In 2003, the VA MS Centers of Excellence created a VA MS National Data Repository as a part of their clinical, administrative, and research mission. This repository, containing over 40,000 unique cases, is one of only two population based repositories for MS in the United States, contains the largest collection of information on men with MS in the world, and is the only national source of information on the care of veterans with MS within VHA.
The original purpose of the VA MS National Data Repository was case identification. It was created from VA demographic, service utilization, pharmacy, and cost databases using broad inclusion criteria in order to locate all individuals receiving care for MS with VHA. However, not every individual coded as receiving care for MS actually has MS. Reasons for misclassification are many, and include the fact that MS has a complex presentation, and the fact that VA encounter/prescription files also contain errors. Finally, recent advancements in the understanding of MS have greatly improved the validity and reliability of MS diagnoses. In 2004, VISN 5 conducted a small preliminary study in which a single nurse practitioner conducted a chart review of cases in that VISN with an ICD-9 code of 340.0 and found that as many as 40% of individuals in the VHA MS National Data Repository would not meet criteria for having definite MS. It also established that an algorithm using readily available patient treatment information could be used to identify 'true' cases of MS. This early work was important in that it underscored the need for case validation within the repository, yet its generalizability was limited due to the fact it was conducted at one time, by a single individual, at a single predominantly-urban location, and without comparison to a face-to-face clinical examination by an MS specialist.
The next logical step in the development of the VA MS National Data Repository would be to continue to identify the best and most efficient methods for validating the cases contained within it. There is a widely accepted gold standard for diagnosing, and confirming a diagnosis of MS, known as the McDonald criteria.4 There is little evidence, however, to suggest that these standards are routinely or uniformly applied to all veterans within VHA. Over the past two years, the VA MS Centers of Excellence have developed an electronic template designed to systematically review these diagnostic criteria for record review or a face-to-face encounter. The feasibility and utility of this electronic template, however, remain to be established. Accurate identification of individuals with a confirmed diagnosis of MS within the repository would increase its clinical value by helping to clarify national treatment needs, its administrative value by providing more accurate estimates of service utilization and cost, and its research value by highlighting individuals who might be recruited for future clinical trials.
OBJECTIVE(S):
Specific Objective 1: Pilot test in VISN 20 the feasibility and utility of a newly-developed electronic template that utilizes an internationally accepted standard of classification of people suspected of having MS. Testing will include:
- Calculating reliability statistics between the electronic template and a face-to-face diagnosis by an MS specialist
- Generating intra- and inter-observer reliability statistics for raters using the template with four different levels of education and professional training.
Specific Objective 2: Create simple electronic queries of existing data within the VA MS National Data Repository that may be used to identify veterans more likely to have MS. Estimate the sensitivity and specificity of these algorithms using template-confirmed diagnosis of MS as the gold standard.
Specific Objective 3: Estimate the total costs associated with validating the entire VA MS National Data Repository.
Specific Objective 4: Validate cases of MS within VISN 20.
METHODS:
Participants will be identified from veterans in the VA MS Data Repository who have received care within VISN 20. Four reviewers including a physiatrist, a nurse, a graduate student, and a research assistant will be trained in the use of
the electronic template. Fifty unique medical records will be assigned to each of the four reviewers. Additionally, each evaluator will review both approximately 25 records that have also been reviewed by another evaluator and approximately 25 records a second time. The reviewers will give a diagnosis using the three McDonald Classifications (1) MS, (2) Possible MS, or (3) Not MS, modified to accommodate chart review. Additionally, a sequential cohort of 25
patients seen by the MS specialist physiatrist will be identified and evaluated in clinic using the McDonald criteria. These 25 veterans will then be randomized to chart review using the electronic template by one of the other reviewers.
Kappa statistics and confidence limits for the outcome variable (diagnosis status) will be calculated using 1) the 25 patients with a diagnoses established by the MS specialist during a clinical visit and record review by the other
reviewers, 2) the 25 records that have been reviewed by all four reviewers, and 3) the 25 records reviewed twice by each reviewer. Sensitivities, specificities and ROC curves of different queries of the VA MS National Data Repository will be
calculated using the results of the electronic chart review to classify veterans as the gold standard.5,6 An example of a possible query might include service connection for MS and one documented inpatient stay for MS.
Cost estimates for validation of the total VA MS Data Repository will be established by 1) determining the optimal level of training necessary for electronic review (part of objective 1), 2) recording completion times for each review, 3) identifying variations in completion time based on demographic and service utilization data contained in the repository (i.e., cases in which the individual is service connected for MS, uses an MS-specific medication, or has had many MSspecific encounters may take less time, on average, to validate) 4) create strata based upon variations in completion time that could be applied to the entire repository, and 5) estimate projected costs for validation of the entire repository based upon product of cost/time unit and total number of time units.
FINDINGS/RESULTS:
No results at this time.
IMPACT:
Results should provide information about the reliability and validity of a chart review template to confirm the diagnosis of MS in the VA MS National Data Repository.
PUBLICATIONS:
None at this time.
DRA:
Chronic Diseases
DRE:
Diagnosis and Prognosis, Epidemiology
Keywords:
Chronic disease (other & unspecified)
MeSH Terms:
none
|
