| Year: 2006 | 
Download Free Adobe Acrobat Reader |
| LHNCBC-2006-057 |
| Besides Precision and Recall: Exploring Alternative Approaches to Evaluating an Automatic Indexing Tool for MEDLINE |
| Neveol A, Zeng K, Bodenreider O |
| Proc AMIA Symp 2006:589-593 |
| This paper explores alternative ap-proaches for the evaluation of an automatic indexing tool for MEDLINE, complementing the traditional precision and recall method. Materials and methods: The performance of MTI, the Medical Text Indexer used at NLM to produce MeSH recommendations for biomedical journal articles is evaluated on a random set of MEDLINE citations. The evaluation examines semantic similarity at the term level (indexing terms). In addition, the documents retrieved by queries resulting from MTI index terms for a given document are compared to the PubMed related citations for this document. Semantic similarity scores between sets of index terms are higher than the corresponding Dice similarity scores. Overall, 75% of the original documents and 58% of the top ten related citations are retrieved by queries based on the automatic indexing. The alternative measures studied in this paper confirm previous findings and may be used to select particular documents from the test set for a more thorough analysis. |
| PDF |
