Using the Guttman scale to define and estimate measurement error in items over time

The case of cognitive decline and the meaning of "points lost"

Rochelle E. Tractenberg, Futoshi Yumoto, Paul S. Aisen, Jeffrey Kaye, Robert J. Mislevy

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

We used a Guttman model to represent responses to test items over time as an approximation of what is often referred to as "points lost" in studies of cognitive decline or interventions. To capture this meaning of "point loss", over four successive assessments, we assumed that once an item is incorrect, it cannot be correct at a later visit. If the loss of a point represents actual decline, then failure of an item to fit the Guttman model over time can be considered measurement error. This representation and definition of measurement error also permits testing the hypotheses that measurement error is constant for items in a test, and that error is independent of "true score", which are two key consequences of the definition of "measurement error" -and thereby, reliability- under Classical Test Theory. We tested the hypotheses by fitting our model to, and comparing our results from, four consecutive annual evaluations in three groups of elderly persons: a) cognitively normal (NC, N = 149); b) diagnosed with possible or probable AD (N = 78); and c) cognitively normal initially and a later diagnosis of AD (converters, N = 133). Of 16 items that converged, error-free measurement of "cognitive loss" was observed for 10 items in NC, eight in converters, and two in AD. We found that measurement error, as we defined it, was inconsistent over time and across cognitive functioning levels, violating the theory underlying reliability and other psychometric characteristics, and key regression assumptions.

Original languageEnglish (US)
Article numbere30019
JournalPLoS One
Volume7
Issue number2
DOIs
StatePublished - Feb 17 2012

Fingerprint

Measurement errors
Delayed Diagnosis
Psychometrics
Reliability theory
testing
Cognitive Dysfunction
Testing

ASJC Scopus subject areas

  • Agricultural and Biological Sciences(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Medicine(all)

Cite this

Using the Guttman scale to define and estimate measurement error in items over time : The case of cognitive decline and the meaning of "points lost". / Tractenberg, Rochelle E.; Yumoto, Futoshi; Aisen, Paul S.; Kaye, Jeffrey; Mislevy, Robert J.

In: PLoS One, Vol. 7, No. 2, e30019, 17.02.2012.

Research output: Contribution to journalArticle

@article{222658fc0c3e4e258e41df0a13c13e69,
title = "Using the Guttman scale to define and estimate measurement error in items over time: The case of cognitive decline and the meaning of {"}points lost{"}",
abstract = "We used a Guttman model to represent responses to test items over time as an approximation of what is often referred to as {"}points lost{"} in studies of cognitive decline or interventions. To capture this meaning of {"}point loss{"}, over four successive assessments, we assumed that once an item is incorrect, it cannot be correct at a later visit. If the loss of a point represents actual decline, then failure of an item to fit the Guttman model over time can be considered measurement error. This representation and definition of measurement error also permits testing the hypotheses that measurement error is constant for items in a test, and that error is independent of {"}true score{"}, which are two key consequences of the definition of {"}measurement error{"} -and thereby, reliability- under Classical Test Theory. We tested the hypotheses by fitting our model to, and comparing our results from, four consecutive annual evaluations in three groups of elderly persons: a) cognitively normal (NC, N = 149); b) diagnosed with possible or probable AD (N = 78); and c) cognitively normal initially and a later diagnosis of AD (converters, N = 133). Of 16 items that converged, error-free measurement of {"}cognitive loss{"} was observed for 10 items in NC, eight in converters, and two in AD. We found that measurement error, as we defined it, was inconsistent over time and across cognitive functioning levels, violating the theory underlying reliability and other psychometric characteristics, and key regression assumptions.",
author = "Tractenberg, {Rochelle E.} and Futoshi Yumoto and Aisen, {Paul S.} and Jeffrey Kaye and Mislevy, {Robert J.}",
year = "2012",
month = "2",
day = "17",
doi = "10.1371/journal.pone.0030019",
language = "English (US)",
volume = "7",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "2",

}

TY - JOUR

T1 - Using the Guttman scale to define and estimate measurement error in items over time

T2 - The case of cognitive decline and the meaning of "points lost"

AU - Tractenberg, Rochelle E.

AU - Yumoto, Futoshi

AU - Aisen, Paul S.

AU - Kaye, Jeffrey

AU - Mislevy, Robert J.

PY - 2012/2/17

Y1 - 2012/2/17

N2 - We used a Guttman model to represent responses to test items over time as an approximation of what is often referred to as "points lost" in studies of cognitive decline or interventions. To capture this meaning of "point loss", over four successive assessments, we assumed that once an item is incorrect, it cannot be correct at a later visit. If the loss of a point represents actual decline, then failure of an item to fit the Guttman model over time can be considered measurement error. This representation and definition of measurement error also permits testing the hypotheses that measurement error is constant for items in a test, and that error is independent of "true score", which are two key consequences of the definition of "measurement error" -and thereby, reliability- under Classical Test Theory. We tested the hypotheses by fitting our model to, and comparing our results from, four consecutive annual evaluations in three groups of elderly persons: a) cognitively normal (NC, N = 149); b) diagnosed with possible or probable AD (N = 78); and c) cognitively normal initially and a later diagnosis of AD (converters, N = 133). Of 16 items that converged, error-free measurement of "cognitive loss" was observed for 10 items in NC, eight in converters, and two in AD. We found that measurement error, as we defined it, was inconsistent over time and across cognitive functioning levels, violating the theory underlying reliability and other psychometric characteristics, and key regression assumptions.

AB - We used a Guttman model to represent responses to test items over time as an approximation of what is often referred to as "points lost" in studies of cognitive decline or interventions. To capture this meaning of "point loss", over four successive assessments, we assumed that once an item is incorrect, it cannot be correct at a later visit. If the loss of a point represents actual decline, then failure of an item to fit the Guttman model over time can be considered measurement error. This representation and definition of measurement error also permits testing the hypotheses that measurement error is constant for items in a test, and that error is independent of "true score", which are two key consequences of the definition of "measurement error" -and thereby, reliability- under Classical Test Theory. We tested the hypotheses by fitting our model to, and comparing our results from, four consecutive annual evaluations in three groups of elderly persons: a) cognitively normal (NC, N = 149); b) diagnosed with possible or probable AD (N = 78); and c) cognitively normal initially and a later diagnosis of AD (converters, N = 133). Of 16 items that converged, error-free measurement of "cognitive loss" was observed for 10 items in NC, eight in converters, and two in AD. We found that measurement error, as we defined it, was inconsistent over time and across cognitive functioning levels, violating the theory underlying reliability and other psychometric characteristics, and key regression assumptions.

UR - http://www.scopus.com/inward/record.url?scp=84857166050&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84857166050&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0030019

DO - 10.1371/journal.pone.0030019

M3 - Article

VL - 7

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 2

M1 - e30019

ER -