Agreement and reliability statistics for shapes

Travis Smith, Ning Smith

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

We describe a methodology for assessing agreement and reliability among a set of shapes. Motivated by recent studies of the reliability of manually segmented medical images, we focus on shapes composed of rasterized, binary-valued data representing closed geometric regions of interest. The methodology naturally generalizes to N dimensions and other data types, though. We formulate the shape variance, shape correlation and shape intraclass correlation coefficient (ICC) in terms of a simple distance metric, the Manhattan norm, which quantifies the absolute difference between any two shapes. We demonstrate applications of this methodology by working through example shape variance calculations in 1-D, for the analysis of overlapping line segments, and 2-D, for the analysis of overlapping regions. We also report the results of a simulated reliability analysis of manually delineated shape boundaries, and we compare the shape ICC with the more conventional and commonly used area ICC. The proposed shape-sensitive methodology captures all of the variation in the shape measurements, and it provides a more accurate estimate of the measurement reliability than an analysis of only the measured areas.

Original languageEnglish (US)
Article numbere0202087
JournalPloS one
Volume13
Issue number8
DOIs
StatePublished - Aug 1 2018

Fingerprint

statistics
Statistics
Reliability analysis
methodology

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)

Cite this

Agreement and reliability statistics for shapes. / Smith, Travis; Smith, Ning.

In: PloS one, Vol. 13, No. 8, e0202087, 01.08.2018.

Research output: Contribution to journalArticle

Smith, Travis ; Smith, Ning. / Agreement and reliability statistics for shapes. In: PloS one. 2018 ; Vol. 13, No. 8.
@article{9ace2bcaf8cd4686bac98241e6c0b783,
title = "Agreement and reliability statistics for shapes",
abstract = "We describe a methodology for assessing agreement and reliability among a set of shapes. Motivated by recent studies of the reliability of manually segmented medical images, we focus on shapes composed of rasterized, binary-valued data representing closed geometric regions of interest. The methodology naturally generalizes to N dimensions and other data types, though. We formulate the shape variance, shape correlation and shape intraclass correlation coefficient (ICC) in terms of a simple distance metric, the Manhattan norm, which quantifies the absolute difference between any two shapes. We demonstrate applications of this methodology by working through example shape variance calculations in 1-D, for the analysis of overlapping line segments, and 2-D, for the analysis of overlapping regions. We also report the results of a simulated reliability analysis of manually delineated shape boundaries, and we compare the shape ICC with the more conventional and commonly used area ICC. The proposed shape-sensitive methodology captures all of the variation in the shape measurements, and it provides a more accurate estimate of the measurement reliability than an analysis of only the measured areas.",
author = "Travis Smith and Ning Smith",
year = "2018",
month = "8",
day = "1",
doi = "10.1371/journal.pone.0202087",
language = "English (US)",
volume = "13",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "8",

}

TY - JOUR

T1 - Agreement and reliability statistics for shapes

AU - Smith, Travis

AU - Smith, Ning

PY - 2018/8/1

Y1 - 2018/8/1

N2 - We describe a methodology for assessing agreement and reliability among a set of shapes. Motivated by recent studies of the reliability of manually segmented medical images, we focus on shapes composed of rasterized, binary-valued data representing closed geometric regions of interest. The methodology naturally generalizes to N dimensions and other data types, though. We formulate the shape variance, shape correlation and shape intraclass correlation coefficient (ICC) in terms of a simple distance metric, the Manhattan norm, which quantifies the absolute difference between any two shapes. We demonstrate applications of this methodology by working through example shape variance calculations in 1-D, for the analysis of overlapping line segments, and 2-D, for the analysis of overlapping regions. We also report the results of a simulated reliability analysis of manually delineated shape boundaries, and we compare the shape ICC with the more conventional and commonly used area ICC. The proposed shape-sensitive methodology captures all of the variation in the shape measurements, and it provides a more accurate estimate of the measurement reliability than an analysis of only the measured areas.

AB - We describe a methodology for assessing agreement and reliability among a set of shapes. Motivated by recent studies of the reliability of manually segmented medical images, we focus on shapes composed of rasterized, binary-valued data representing closed geometric regions of interest. The methodology naturally generalizes to N dimensions and other data types, though. We formulate the shape variance, shape correlation and shape intraclass correlation coefficient (ICC) in terms of a simple distance metric, the Manhattan norm, which quantifies the absolute difference between any two shapes. We demonstrate applications of this methodology by working through example shape variance calculations in 1-D, for the analysis of overlapping line segments, and 2-D, for the analysis of overlapping regions. We also report the results of a simulated reliability analysis of manually delineated shape boundaries, and we compare the shape ICC with the more conventional and commonly used area ICC. The proposed shape-sensitive methodology captures all of the variation in the shape measurements, and it provides a more accurate estimate of the measurement reliability than an analysis of only the measured areas.

UR - http://www.scopus.com/inward/record.url?scp=85052075673&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052075673&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0202087

DO - 10.1371/journal.pone.0202087

M3 - Article

C2 - 30138326

AN - SCOPUS:85052075673

VL - 13

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 8

M1 - e0202087

ER -