Relationship of subjective ratings of image quality and observer performance

Howard E. Rockette, Christopher M. Johns, Jane Weissman, J. Michael Holbert, Jules H. Sumkin, Jill L. King, David Gur

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

The relationship between radiologists' perception of image quality and their actual performance was assessed. If the two variables are strongly correlated, the easier obtained perception of quality index might be used as a prerequisite test to determine if a ROC study is justified. One-hundred seventy cases were evaluated for the presence or absence of interstitial disease and nodules by nine readers using seven display modes. Each reader also assigned each image a mode- specific perceived quality rating using a 5-category ordinal scale. Average perceived quality was highest for conventional film. It was slightly poorer for the subsets of cases with interstitial disease and for cases classified independently as 'subtle.' Trend tests indicated a relationship between area under the ROC curves (Az) and perceived image quality for nodules. For interstitial disease, the relationship was weaker and of borderline statistical significance. The subjective image quality index was related to the area under the ROC curve, but the average difference between pairs of display modes was not a good predictor of difference in actual observer performances. A subjective quality index may have limited usefulness in screening differences between modalities prior to the performance of a ROC study.

Original languageEnglish (US)
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
Pages152-159
Number of pages8
Volume3036
DOIs
StatePublished - 1997
Externally publishedYes
EventMedical Imaging 1997: Image Perception - Newport Beach, CA, United States
Duration: Feb 26 1997Feb 26 1997

Other

OtherMedical Imaging 1997: Image Perception
CountryUnited States
CityNewport Beach, CA
Period2/26/972/26/97

Fingerprint

ratings
Image Quality
Image quality
Observer
Nodule
Receiver Operating Characteristic Curve
Display devices
Display
Trend Test
interstitials
Ordinal Scale
nodules
readers
Statistical Significance
Screening
Modality
Predictors
curves
Subset
Relationships

Keywords

  • Diagnosis
  • Image quality
  • Observer performance
  • Receiver Operating Characteristic (ROC)

ASJC Scopus subject areas

  • Applied Mathematics
  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics

Cite this

Rockette, H. E., Johns, C. M., Weissman, J., Holbert, J. M., Sumkin, J. H., King, J. L., & Gur, D. (1997). Relationship of subjective ratings of image quality and observer performance. In Proceedings of SPIE - The International Society for Optical Engineering (Vol. 3036, pp. 152-159) https://doi.org/10.1117/12.271287

Relationship of subjective ratings of image quality and observer performance. / Rockette, Howard E.; Johns, Christopher M.; Weissman, Jane; Holbert, J. Michael; Sumkin, Jules H.; King, Jill L.; Gur, David.

Proceedings of SPIE - The International Society for Optical Engineering. Vol. 3036 1997. p. 152-159.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Rockette, HE, Johns, CM, Weissman, J, Holbert, JM, Sumkin, JH, King, JL & Gur, D 1997, Relationship of subjective ratings of image quality and observer performance. in Proceedings of SPIE - The International Society for Optical Engineering. vol. 3036, pp. 152-159, Medical Imaging 1997: Image Perception, Newport Beach, CA, United States, 2/26/97. https://doi.org/10.1117/12.271287
Rockette HE, Johns CM, Weissman J, Holbert JM, Sumkin JH, King JL et al. Relationship of subjective ratings of image quality and observer performance. In Proceedings of SPIE - The International Society for Optical Engineering. Vol. 3036. 1997. p. 152-159 https://doi.org/10.1117/12.271287
Rockette, Howard E. ; Johns, Christopher M. ; Weissman, Jane ; Holbert, J. Michael ; Sumkin, Jules H. ; King, Jill L. ; Gur, David. / Relationship of subjective ratings of image quality and observer performance. Proceedings of SPIE - The International Society for Optical Engineering. Vol. 3036 1997. pp. 152-159
@inproceedings{94c83b8995004b06a849ebd1c2e52f6c,
title = "Relationship of subjective ratings of image quality and observer performance",
abstract = "The relationship between radiologists' perception of image quality and their actual performance was assessed. If the two variables are strongly correlated, the easier obtained perception of quality index might be used as a prerequisite test to determine if a ROC study is justified. One-hundred seventy cases were evaluated for the presence or absence of interstitial disease and nodules by nine readers using seven display modes. Each reader also assigned each image a mode- specific perceived quality rating using a 5-category ordinal scale. Average perceived quality was highest for conventional film. It was slightly poorer for the subsets of cases with interstitial disease and for cases classified independently as 'subtle.' Trend tests indicated a relationship between area under the ROC curves (Az) and perceived image quality for nodules. For interstitial disease, the relationship was weaker and of borderline statistical significance. The subjective image quality index was related to the area under the ROC curve, but the average difference between pairs of display modes was not a good predictor of difference in actual observer performances. A subjective quality index may have limited usefulness in screening differences between modalities prior to the performance of a ROC study.",
keywords = "Diagnosis, Image quality, Observer performance, Receiver Operating Characteristic (ROC)",
author = "Rockette, {Howard E.} and Johns, {Christopher M.} and Jane Weissman and Holbert, {J. Michael} and Sumkin, {Jules H.} and King, {Jill L.} and David Gur",
year = "1997",
doi = "10.1117/12.271287",
language = "English (US)",
volume = "3036",
pages = "152--159",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",

}

TY - GEN

T1 - Relationship of subjective ratings of image quality and observer performance

AU - Rockette, Howard E.

AU - Johns, Christopher M.

AU - Weissman, Jane

AU - Holbert, J. Michael

AU - Sumkin, Jules H.

AU - King, Jill L.

AU - Gur, David

PY - 1997

Y1 - 1997

N2 - The relationship between radiologists' perception of image quality and their actual performance was assessed. If the two variables are strongly correlated, the easier obtained perception of quality index might be used as a prerequisite test to determine if a ROC study is justified. One-hundred seventy cases were evaluated for the presence or absence of interstitial disease and nodules by nine readers using seven display modes. Each reader also assigned each image a mode- specific perceived quality rating using a 5-category ordinal scale. Average perceived quality was highest for conventional film. It was slightly poorer for the subsets of cases with interstitial disease and for cases classified independently as 'subtle.' Trend tests indicated a relationship between area under the ROC curves (Az) and perceived image quality for nodules. For interstitial disease, the relationship was weaker and of borderline statistical significance. The subjective image quality index was related to the area under the ROC curve, but the average difference between pairs of display modes was not a good predictor of difference in actual observer performances. A subjective quality index may have limited usefulness in screening differences between modalities prior to the performance of a ROC study.

AB - The relationship between radiologists' perception of image quality and their actual performance was assessed. If the two variables are strongly correlated, the easier obtained perception of quality index might be used as a prerequisite test to determine if a ROC study is justified. One-hundred seventy cases were evaluated for the presence or absence of interstitial disease and nodules by nine readers using seven display modes. Each reader also assigned each image a mode- specific perceived quality rating using a 5-category ordinal scale. Average perceived quality was highest for conventional film. It was slightly poorer for the subsets of cases with interstitial disease and for cases classified independently as 'subtle.' Trend tests indicated a relationship between area under the ROC curves (Az) and perceived image quality for nodules. For interstitial disease, the relationship was weaker and of borderline statistical significance. The subjective image quality index was related to the area under the ROC curve, but the average difference between pairs of display modes was not a good predictor of difference in actual observer performances. A subjective quality index may have limited usefulness in screening differences between modalities prior to the performance of a ROC study.

KW - Diagnosis

KW - Image quality

KW - Observer performance

KW - Receiver Operating Characteristic (ROC)

UR - http://www.scopus.com/inward/record.url?scp=0006787941&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0006787941&partnerID=8YFLogxK

U2 - 10.1117/12.271287

DO - 10.1117/12.271287

M3 - Conference contribution

VL - 3036

SP - 152

EP - 159

BT - Proceedings of SPIE - The International Society for Optical Engineering

ER -