Comparing the quality of accessing the medical literature using content? Based visual and textual information retrieval

Henning Müller, Jayashree Kalpathy-Cramer, Charles E. Kahn, William (Bill) Hersh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

Content-based visual information (or image) retrieval (CBIR) has been an extremely active research domain within medical imaging over the past ten years, with the goal of improving the management of visual medical information. Many technical solutions have been proposed, and application scenarios for image retrieval as well as image classification have been set up. However, in contrast to medical information retrieval using textual methods, visual retrieval has only rarely been applied in clinical practice. This is despite the large amount and variety of visual information produced in hospitals every day. This information overload imposes a significant burden upon clinicians, and CBIR technologies have the potential to help the situation. However, in order for CBIR to become an accepted clinical tool, it must demonstrate a higher level of technical maturity than it has to date. Since 2004, the ImageCLEF benchmark has included a task for the comparison of visual information retrieval algorithms for medical applications. In 2005, a task for medical image classification was introduced and both tasks have been run successfully for the past four years. These benchmarks allow an annual comparison of visual retrieval techniques based on the same data sets and the same query tasks, enabling the meaningful comparison of various retrieval techniques. The datasets used from 2004?2007 contained images and annotations from medical teaching files. In 2008, however, the dataset used was made up of 67,000 images (along with their associated figure captions and the full text of their corresponding articles) from two Radiological Society of North America (RSNA) scientific journals. This article describes the results of the medical image retrieval task of the ImageCLEF 2008 evaluation campaign. We compare the retrieval results of both visual and textual information retrieval systems from 15 research groups on the aforementioned data set. The results show clearly that, currently, visual retrieval alone does not achieve the performance necessary for real?world clinical applications. Most of the common visual retrieval techniques have a MAP (Mean Average Precision) of around 2-3%, which is much lower than that achieved using textual retrieval (MAP=29%). Advanced machine learning techniques, together with good training data, have been shown to improve the performance of visual retrieval systems in the past. Multimodal retrieval (basing retrieval on both visual and textual information) can achieve better results than purely visual, but only when carefully applied. In many cases, multimodal retrieval systems performed even worse than purely textual retrieval systems. On the other hand, some multimodal retrieval systems demonstrated significantly increased early precision, which has been shown to be a desirable behavior in real?world systems.

Original languageEnglish (US)
Title of host publicationProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume7264
DOIs
StatePublished - 2009
EventMedical Imaging 2009: Advanced PACS-based Imaging Informatics and Therapeutic Applications - Lake Buena Vista, FL, United States
Duration: Feb 11 2009Feb 12 2009

Other

OtherMedical Imaging 2009: Advanced PACS-based Imaging Informatics and Therapeutic Applications
CountryUnited States
CityLake Buena Vista, FL
Period2/11/092/12/09

Fingerprint

information retrieval
Information Storage and Retrieval
Image retrieval
Information retrieval
retrieval
Image classification
Benchmarking
Information retrieval systems
Medical imaging
Medical applications
Learning systems
Teaching
Diagnostic Imaging
Research
Information Systems
Technology
image classification
Datasets
education
annotations

Keywords

  • Content-based image retrieval
  • Evaluation
  • Information retrieval
  • Medical image retrieval
  • Multimodal information search
  • Scientific literature

ASJC Scopus subject areas

  • Atomic and Molecular Physics, and Optics
  • Electronic, Optical and Magnetic Materials
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Cite this

Müller, H., Kalpathy-Cramer, J., Kahn, C. E., & Hersh, W. B. (2009). Comparing the quality of accessing the medical literature using content? Based visual and textual information retrieval. In Progress in Biomedical Optics and Imaging - Proceedings of SPIE (Vol. 7264). [726405] https://doi.org/10.1117/12.811416

Comparing the quality of accessing the medical literature using content? Based visual and textual information retrieval. / Müller, Henning; Kalpathy-Cramer, Jayashree; Kahn, Charles E.; Hersh, William (Bill).

Progress in Biomedical Optics and Imaging - Proceedings of SPIE. Vol. 7264 2009. 726405.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Müller, H, Kalpathy-Cramer, J, Kahn, CE & Hersh, WB 2009, Comparing the quality of accessing the medical literature using content? Based visual and textual information retrieval. in Progress in Biomedical Optics and Imaging - Proceedings of SPIE. vol. 7264, 726405, Medical Imaging 2009: Advanced PACS-based Imaging Informatics and Therapeutic Applications, Lake Buena Vista, FL, United States, 2/11/09. https://doi.org/10.1117/12.811416
Müller H, Kalpathy-Cramer J, Kahn CE, Hersh WB. Comparing the quality of accessing the medical literature using content? Based visual and textual information retrieval. In Progress in Biomedical Optics and Imaging - Proceedings of SPIE. Vol. 7264. 2009. 726405 https://doi.org/10.1117/12.811416
Müller, Henning ; Kalpathy-Cramer, Jayashree ; Kahn, Charles E. ; Hersh, William (Bill). / Comparing the quality of accessing the medical literature using content? Based visual and textual information retrieval. Progress in Biomedical Optics and Imaging - Proceedings of SPIE. Vol. 7264 2009.
@inproceedings{7894cfc84b6a4d96b18f594d65873077,
title = "Comparing the quality of accessing the medical literature using content? Based visual and textual information retrieval",
abstract = "Content-based visual information (or image) retrieval (CBIR) has been an extremely active research domain within medical imaging over the past ten years, with the goal of improving the management of visual medical information. Many technical solutions have been proposed, and application scenarios for image retrieval as well as image classification have been set up. However, in contrast to medical information retrieval using textual methods, visual retrieval has only rarely been applied in clinical practice. This is despite the large amount and variety of visual information produced in hospitals every day. This information overload imposes a significant burden upon clinicians, and CBIR technologies have the potential to help the situation. However, in order for CBIR to become an accepted clinical tool, it must demonstrate a higher level of technical maturity than it has to date. Since 2004, the ImageCLEF benchmark has included a task for the comparison of visual information retrieval algorithms for medical applications. In 2005, a task for medical image classification was introduced and both tasks have been run successfully for the past four years. These benchmarks allow an annual comparison of visual retrieval techniques based on the same data sets and the same query tasks, enabling the meaningful comparison of various retrieval techniques. The datasets used from 2004?2007 contained images and annotations from medical teaching files. In 2008, however, the dataset used was made up of 67,000 images (along with their associated figure captions and the full text of their corresponding articles) from two Radiological Society of North America (RSNA) scientific journals. This article describes the results of the medical image retrieval task of the ImageCLEF 2008 evaluation campaign. We compare the retrieval results of both visual and textual information retrieval systems from 15 research groups on the aforementioned data set. The results show clearly that, currently, visual retrieval alone does not achieve the performance necessary for real?world clinical applications. Most of the common visual retrieval techniques have a MAP (Mean Average Precision) of around 2-3{\%}, which is much lower than that achieved using textual retrieval (MAP=29{\%}). Advanced machine learning techniques, together with good training data, have been shown to improve the performance of visual retrieval systems in the past. Multimodal retrieval (basing retrieval on both visual and textual information) can achieve better results than purely visual, but only when carefully applied. In many cases, multimodal retrieval systems performed even worse than purely textual retrieval systems. On the other hand, some multimodal retrieval systems demonstrated significantly increased early precision, which has been shown to be a desirable behavior in real?world systems.",
keywords = "Content-based image retrieval, Evaluation, Information retrieval, Medical image retrieval, Multimodal information search, Scientific literature",
author = "Henning M{\"u}ller and Jayashree Kalpathy-Cramer and Kahn, {Charles E.} and Hersh, {William (Bill)}",
year = "2009",
doi = "10.1117/12.811416",
language = "English (US)",
isbn = "9780819475152",
volume = "7264",
booktitle = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",

}

TY - GEN

T1 - Comparing the quality of accessing the medical literature using content? Based visual and textual information retrieval

AU - Müller, Henning

AU - Kalpathy-Cramer, Jayashree

AU - Kahn, Charles E.

AU - Hersh, William (Bill)

PY - 2009

Y1 - 2009

N2 - Content-based visual information (or image) retrieval (CBIR) has been an extremely active research domain within medical imaging over the past ten years, with the goal of improving the management of visual medical information. Many technical solutions have been proposed, and application scenarios for image retrieval as well as image classification have been set up. However, in contrast to medical information retrieval using textual methods, visual retrieval has only rarely been applied in clinical practice. This is despite the large amount and variety of visual information produced in hospitals every day. This information overload imposes a significant burden upon clinicians, and CBIR technologies have the potential to help the situation. However, in order for CBIR to become an accepted clinical tool, it must demonstrate a higher level of technical maturity than it has to date. Since 2004, the ImageCLEF benchmark has included a task for the comparison of visual information retrieval algorithms for medical applications. In 2005, a task for medical image classification was introduced and both tasks have been run successfully for the past four years. These benchmarks allow an annual comparison of visual retrieval techniques based on the same data sets and the same query tasks, enabling the meaningful comparison of various retrieval techniques. The datasets used from 2004?2007 contained images and annotations from medical teaching files. In 2008, however, the dataset used was made up of 67,000 images (along with their associated figure captions and the full text of their corresponding articles) from two Radiological Society of North America (RSNA) scientific journals. This article describes the results of the medical image retrieval task of the ImageCLEF 2008 evaluation campaign. We compare the retrieval results of both visual and textual information retrieval systems from 15 research groups on the aforementioned data set. The results show clearly that, currently, visual retrieval alone does not achieve the performance necessary for real?world clinical applications. Most of the common visual retrieval techniques have a MAP (Mean Average Precision) of around 2-3%, which is much lower than that achieved using textual retrieval (MAP=29%). Advanced machine learning techniques, together with good training data, have been shown to improve the performance of visual retrieval systems in the past. Multimodal retrieval (basing retrieval on both visual and textual information) can achieve better results than purely visual, but only when carefully applied. In many cases, multimodal retrieval systems performed even worse than purely textual retrieval systems. On the other hand, some multimodal retrieval systems demonstrated significantly increased early precision, which has been shown to be a desirable behavior in real?world systems.

AB - Content-based visual information (or image) retrieval (CBIR) has been an extremely active research domain within medical imaging over the past ten years, with the goal of improving the management of visual medical information. Many technical solutions have been proposed, and application scenarios for image retrieval as well as image classification have been set up. However, in contrast to medical information retrieval using textual methods, visual retrieval has only rarely been applied in clinical practice. This is despite the large amount and variety of visual information produced in hospitals every day. This information overload imposes a significant burden upon clinicians, and CBIR technologies have the potential to help the situation. However, in order for CBIR to become an accepted clinical tool, it must demonstrate a higher level of technical maturity than it has to date. Since 2004, the ImageCLEF benchmark has included a task for the comparison of visual information retrieval algorithms for medical applications. In 2005, a task for medical image classification was introduced and both tasks have been run successfully for the past four years. These benchmarks allow an annual comparison of visual retrieval techniques based on the same data sets and the same query tasks, enabling the meaningful comparison of various retrieval techniques. The datasets used from 2004?2007 contained images and annotations from medical teaching files. In 2008, however, the dataset used was made up of 67,000 images (along with their associated figure captions and the full text of their corresponding articles) from two Radiological Society of North America (RSNA) scientific journals. This article describes the results of the medical image retrieval task of the ImageCLEF 2008 evaluation campaign. We compare the retrieval results of both visual and textual information retrieval systems from 15 research groups on the aforementioned data set. The results show clearly that, currently, visual retrieval alone does not achieve the performance necessary for real?world clinical applications. Most of the common visual retrieval techniques have a MAP (Mean Average Precision) of around 2-3%, which is much lower than that achieved using textual retrieval (MAP=29%). Advanced machine learning techniques, together with good training data, have been shown to improve the performance of visual retrieval systems in the past. Multimodal retrieval (basing retrieval on both visual and textual information) can achieve better results than purely visual, but only when carefully applied. In many cases, multimodal retrieval systems performed even worse than purely textual retrieval systems. On the other hand, some multimodal retrieval systems demonstrated significantly increased early precision, which has been shown to be a desirable behavior in real?world systems.

KW - Content-based image retrieval

KW - Evaluation

KW - Information retrieval

KW - Medical image retrieval

KW - Multimodal information search

KW - Scientific literature

UR - http://www.scopus.com/inward/record.url?scp=67149099668&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67149099668&partnerID=8YFLogxK

U2 - 10.1117/12.811416

DO - 10.1117/12.811416

M3 - Conference contribution

AN - SCOPUS:67149099668

SN - 9780819475152

VL - 7264

BT - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

ER -