Automatic image modality based classification and annotation to improve medical image retrieval

Jayashree Kalpathy-Cramer, William Hersh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

47 Scopus citations


Medical image retrieval can play an important role for diagnostic and teaching purposes in medicine. Image modality is an important visual characteristic that can be used to improve retrieval performance. Many test and online collections do not contain information about the image modality. We have created an automatic image classifier for both grey-scale and colour medical images. We evaluated the performance of the two modality classifiers, one for grey-scale images and the other for colour images on the CISMeF and the ImageCLEFmed 2006 databases. Both classifiers were created using a neural network architecture for learning. Low level colour and texture based feature vectors were extracted to train the network. Both classifiers achieved an accuracy of > 95% on the test collections that they were tested on. We also evaluated the performance of these classifiers on a selection of queries from the ImageCLEFmed 2006. The precision of the results was improved by using the modality classifier to resort the results of a textual query.

Original languageEnglish (US)
Title of host publicationMEDINFO 2007 - Proceedings of the 12th World Congress on Health (Medical) Informatics
Subtitle of host publicationBuilding Sustainable Health Systems
PublisherIOS Press
Number of pages5
ISBN (Print)9781586037741
StatePublished - 2007
Event12th World Congress on Medical Informatics, MEDINFO 2007 - Brisbane, QLD, Australia
Duration: Aug 20 2007Aug 24 2007

Publication series

NameStudies in Health Technology and Informatics
ISSN (Print)0926-9630
ISSN (Electronic)1879-8365


Other12th World Congress on Medical Informatics, MEDINFO 2007
CityBrisbane, QLD


  • content-based image retrieval
  • image annotation
  • medical imaging
  • neural networks

ASJC Scopus subject areas

  • Biomedical Engineering
  • Health Informatics
  • Health Information Management


Dive into the research topics of 'Automatic image modality based classification and annotation to improve medical image retrieval'. Together they form a unique fingerprint.

Cite this