Consortium of Healthcare Informatics Research: Translational Use Case Projects
Mary K. Goldstein MD MS
VA Palo Alto Health Care System, Palo Alto, CA
Palo Alto, CA
Funding Period: February 2009 - September 2013

BACKGROUND/RATIONALE:
The mission of the Consortium for Healthcare Informatics Research (CHIR) is to improve the health of veterans through foundational and applied informatics research to advance the effective use of unstructured text in the electronic health record.

OBJECTIVE(S):
The CHIR Translational Use Case Projects (TUCPs) aim to assess the capability for rapid development of natural language processing (NLP) to topics of high clinical-quality importance to the VA. The TUCPs apply information extraction techniques to identify and resolve issues, providing early experience for CHIR in practical issues such as reference standard annotations and use of the secure VINCI data resource. Sequential Rounds of TUCPs build on other work of CHIR.

METHODS:
Each TUCP develops its own algorithms for text-abstraction. Typically, projects include mapping key concepts in text to a standardized vocabulary suitable to the clinical domain. Lexicons are refined as necessary to include synonyms, abbreviations, and common spellings of key words. After algorithm development, clinically useful metrics will be computed from the text (e.g. % of patients with EF < 40; number of lymph nodes examined and number positive for tumor). The text-abstraction findings from each algorithm are compared with a reference standard annotation, that is, manually marked records that indicate text that should be identified by text-processing algorithms, by trained annotators using annotation schemata prepared through field testing. These records form an annotated corpus of reports that will be used to test the NLP tools' accuracy and precision. Several rounds of TUCPs address text-extraction for VA clinical/quality high-priority areas and/or extend successful NLP to move closer to wide application to VA data.

FINDINGS/RESULTS:
(1) The Lymph Node (LN) project developed Automated Retrieval Console (ARC), an open source software to improve the process of information retrieval. The algorithm identified LN's examined and LN's positive for cancer with Recall 0.96 for both and precision 0.94 and 0.95 respectively. (2) The Ejection Fraction (EF) system was developed; test results recall (sensitivity) 98%, precision (pos pred value) 100%, F-measure 0.992 in classifying EF<40. (3) The Chest X-Ray (NLP) has recall 95% and precision 98% for line/device mentions on application to a new set of 500 reports. (4) The Contraception project has developed an annotation schema, ontology and NLP system. The annotation schema was applied to 1,739 text notes for 227 female Veteran patients. The ontology identified 84 (out of 1,739) notes with contraception terms, 52 (of 84) notes that had multiple terms and 7 (of 84) terms negated. (5) The Falls project applied a data/text mining-based approach and successfully created reasonable classifiers that are easily interpretable and can serve as a base for handcrafted expert refinements.

IMPACT:
The tools being developed have potential for automating components of quality assessment to free time of human quality reviewers to focus on questions requiring human judgment. For example: The tool to extract EF from echocardiography reports has potential to contribute to automation of important performance measures for care of patients with heart failure. The extraction of line and device information from CXR reports has potential to contribute to automation of counts of "line-days" which is an important measure for infection surveillance.

PUBLICATIONS:

Journal Articles

1. Garvin JH, DuVall SL, South BR, Bray BE, Bolton D, Heavirland J, Pickard S, Heidenreich P, Shen S, Weir C, Samore M, Goldstein MK. Automated extraction of ejection fraction for quality measurement using regular expressions in Unstructured Information Management Architecture (UIMA) for heart failure. Journal of the American Medical Informatics Association : JAMIA. 2012 Sep 1; 19:(5):859-66.
2. Hope CJ, Garvin JH, Sauer BC. Information extraction from narrative data. American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists. 2012 Mar 15; 69(6):455, 460-1.
3. Garvin JH, Elkin PL, Shen S, Brown S, Trusko B, Wang E, Hoke L, Quiaoit Y, Lajoie J, Weiner MG, Graham P, Speroff T. Automated Quality Measurement in Department of the Veterans Affairs Discharge Instructions for Patients with Congestive Heart Failure. Journal For Healthcare Quality. 2012 Jan 31.
4. Rubin D, Wang D, Chambers DA, Chambers JG, South BR, Goldstein MK. Natural language processing for lines and devices in portable chest x-rays. AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium. 2010 Nov 13; 2010:692-6.

Conference Presentations

1. Wang D, Rubin DL, Chambers DA, South BR, Hwang TS, Goldstein MK. Using Natural Language Processing to Identify Lines and Devices in Portable Chest X-Ray Reports. Poster session presented at: Bay Area Clinical Research Annual Symposium; 2011 Nov 4; San Francisco, CA.
2. Hope CJ, Garvin JH, Gundlapalli AV. Incomplete and selective Documentation of delirium in the VA Electronic medical Record. Poster session presented at: American Medical Informatics Association Annual Symposium; 2011 Oct 23; Washington, DC.
3. Goldstein MK. CHIR and Collaboration: Risk and Benefits of Collaboration. Paper presented at: VA Consortium for Healthcare Informatics Research Steering Committee Meeting; 2011 Oct 17; Palo Alto, CA.
4. Goldstein MK. Power Text Data: Examples from Performance and Quality Measurement. Paper presented at: VA Consortium for Healthcare Informatics Research Steering Committee Meeting; 2011 Oct 17; Palo Alto, CA.
5. Garvin JH, South B, Bolton DJ, Shen S, Samore MH, DuVall SL. Automated extraction of ejection fraction (EF) for Heart Failure (HF) from VA Echocardiogram reports. Poster session presented at: VA HSR&D National Meeting; 2011 Feb 18; National Harbor, MD.
6. Rubin DL, Wang D, Chambers DA, Chambers J, South B, Goldstein MK. Extracting Free Text from Electronic Health Record: Natural Language Processing to Identify Lines and Devices in Portable Chest X-Ray Reports. Poster session presented at: VA HSR&D National Meeting; 2011 Feb 17; National Harbor, MD.
7. Goldstein MK. HSR&D Future Directions II: Medical Informatics. Paper presented at: VA HSR&D Career Development Annual Meeting; 2010 Feb 26; San Francisco, CA.
8. D'Avolio LW, South B, Shen S, Garvin JH, Goldstein MK. Reducing Dependency on Manual Chart Review through Automated Information Extraction Methods. Poster session presented at: VA HSR&D National Meeting; 2009 Feb 12; Baltimore, MD.

Center Product

1. Goldstein MK, Garvin J, Meystre S. Developing Applied Informatics Information Extraction Tools in VA: CUIMANDREef (Capture with UIMA of Needed Data using Regular Expressions for EF) and CHIEF (Congestive Heart Failure Information Extraction Framework). HSR&D Cyber Seminars on CHIR (Consortium for Healthcare Informatics Research). [Cyberseminar]. 2011 Jun 30.
2. Goldstein MK. Automated Detection of Lines/Devices from Chest Radiograph Reports: CXR Translational Use Case Project. [Cyberseminar]. 2010 Jun 15.
3. Goldstein MK. Potential for CHIR in QUERI and HSR&D Projects. [Cyberseminar]. 2010 Apr 1.

DRA: Health Systems, Cardiovascular Disease
DRE: Diagnosis, Treatment - Comparative Effectiveness
Keywords: Data Management, Decision Support, Healthcare Algorithms, Information Management, Knowledge Integration, Medication Management, Natural Language Processing, Surveillance
MeSH Terms: none