Automatic image modality based classification and annotation to improve medical image retrieval

Jayashree Kalpathy-Cramer, William (Bill) Hersh

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Medical image retrieval can play an important role for diagnostic and teaching purposes in medicine. Image modality is an important visual characteristic that can be used to improve retrieval performance. Many test and online collections do not contain information about the image modality. We have created an automatic image classifier for both grey-scale and colour medical images. We evaluated the performance of the two modality classifiers, one for grey-scale images and the other for colour images on the CISMeF and the ImageCLEFmed 2006 databases. Both classifiers were created using a neural network architecture for learning. Low level colour and texture based feature vectors were extracted to train the network. Both classifiers achieved an accuracy of > 95% on the test collections that they were tested on. We also evaluated the performance of these classifiers on a selection of queries from the ImageCLEFmed 2006. The precision of the results was improved by using the modality classifier to resort the results of a textual query.

Original languageEnglish (US)
Title of host publicationStudies in Health Technology and Informatics
Number of pages5
StatePublished - 2007
Event12th World Congress on Medical Informatics, MEDINFO 2007 - Brisbane, QLD, Australia
Duration: Aug 20 2007Aug 24 2007


Other12th World Congress on Medical Informatics, MEDINFO 2007
CityBrisbane, QLD


  • content-based image retrieval
  • image annotation
  • medical imaging
  • neural networks

ASJC Scopus subject areas

  • Biomedical Engineering
  • Health Informatics
  • Health Information Management


Dive into the research topics of 'Automatic image modality based classification and annotation to improve medical image retrieval'. Together they form a unique fingerprint.

Cite this