Reproducibility and responsiveness of health status measures statistics and strategies for evaluation

Richard (Rick) Deyo, Paula Diehr, Donald L. Patrick

Research output: Contribution to journalArticle

1091 Citations (Scopus)

Abstract

Before being introduced to wide use, health status instruments should be evaluated for reliability and validity. Increasingly, they are also tested for responsiveness to important clinical changes. Although standards exist for assessing these properties, confusion and inconsistency arise because multiple statistics are used for the same property; controversy exists over how to measure responsiveness; many statistics are unavailable on common software programs; strategies for measuring these properties vary; and it is often unclear how to define a clinically important change in patient status. Using data from a clinical trial of therapy for back pain, we demonstrate the calculation of several statistics for measuring reproducibility and responsiveness, and demonstrate relationships among them. Simple computational guides for several statistics are provided. We conclude that reproducibility should generally be quantified with the intraclass correlation coefficient rather than the more common Pearson r. Assessing reproducibility by retest at one-to-two week intervals (rather than a shorter interval) may result in more realistic estimates of the variability to be observed among control subjects in a longitudinal study. Instrument responsiveness should be quantified using indicators of effect size, a modified effect size statistic proposed by Guyatt, or the use of receiver operating characteristic (ROC) curves to describe how well various score changes can distinguish improved from unimproved patients.

Original languageEnglish (US)
JournalControlled Clinical Trials
Volume12
Issue number4 SUPPL.
DOIs
StatePublished - 1991
Externally publishedYes

Fingerprint

Health Status
Back Pain
Reproducibility of Results
ROC Curve
Longitudinal Studies
Software
Clinical Trials
Therapeutics

Keywords

  • functional status
  • Health status
  • quality-of-life
  • questionnaires
  • responsiveness

ASJC Scopus subject areas

  • Pharmacology

Cite this

Reproducibility and responsiveness of health status measures statistics and strategies for evaluation. / Deyo, Richard (Rick); Diehr, Paula; Patrick, Donald L.

In: Controlled Clinical Trials, Vol. 12, No. 4 SUPPL., 1991.

Research output: Contribution to journalArticle

@article{1e7101c1e9514690b5098952b0cea7ec,
title = "Reproducibility and responsiveness of health status measures statistics and strategies for evaluation",
abstract = "Before being introduced to wide use, health status instruments should be evaluated for reliability and validity. Increasingly, they are also tested for responsiveness to important clinical changes. Although standards exist for assessing these properties, confusion and inconsistency arise because multiple statistics are used for the same property; controversy exists over how to measure responsiveness; many statistics are unavailable on common software programs; strategies for measuring these properties vary; and it is often unclear how to define a clinically important change in patient status. Using data from a clinical trial of therapy for back pain, we demonstrate the calculation of several statistics for measuring reproducibility and responsiveness, and demonstrate relationships among them. Simple computational guides for several statistics are provided. We conclude that reproducibility should generally be quantified with the intraclass correlation coefficient rather than the more common Pearson r. Assessing reproducibility by retest at one-to-two week intervals (rather than a shorter interval) may result in more realistic estimates of the variability to be observed among control subjects in a longitudinal study. Instrument responsiveness should be quantified using indicators of effect size, a modified effect size statistic proposed by Guyatt, or the use of receiver operating characteristic (ROC) curves to describe how well various score changes can distinguish improved from unimproved patients.",
keywords = "functional status, Health status, quality-of-life, questionnaires, responsiveness",
author = "Deyo, {Richard (Rick)} and Paula Diehr and Patrick, {Donald L.}",
year = "1991",
doi = "10.1016/S0197-2456(05)80019-4",
language = "English (US)",
volume = "12",
journal = "Controlled Clinical Trials",
issn = "0197-2456",
publisher = "Elsevier BV",
number = "4 SUPPL.",

}

TY - JOUR

T1 - Reproducibility and responsiveness of health status measures statistics and strategies for evaluation

AU - Deyo, Richard (Rick)

AU - Diehr, Paula

AU - Patrick, Donald L.

PY - 1991

Y1 - 1991

N2 - Before being introduced to wide use, health status instruments should be evaluated for reliability and validity. Increasingly, they are also tested for responsiveness to important clinical changes. Although standards exist for assessing these properties, confusion and inconsistency arise because multiple statistics are used for the same property; controversy exists over how to measure responsiveness; many statistics are unavailable on common software programs; strategies for measuring these properties vary; and it is often unclear how to define a clinically important change in patient status. Using data from a clinical trial of therapy for back pain, we demonstrate the calculation of several statistics for measuring reproducibility and responsiveness, and demonstrate relationships among them. Simple computational guides for several statistics are provided. We conclude that reproducibility should generally be quantified with the intraclass correlation coefficient rather than the more common Pearson r. Assessing reproducibility by retest at one-to-two week intervals (rather than a shorter interval) may result in more realistic estimates of the variability to be observed among control subjects in a longitudinal study. Instrument responsiveness should be quantified using indicators of effect size, a modified effect size statistic proposed by Guyatt, or the use of receiver operating characteristic (ROC) curves to describe how well various score changes can distinguish improved from unimproved patients.

AB - Before being introduced to wide use, health status instruments should be evaluated for reliability and validity. Increasingly, they are also tested for responsiveness to important clinical changes. Although standards exist for assessing these properties, confusion and inconsistency arise because multiple statistics are used for the same property; controversy exists over how to measure responsiveness; many statistics are unavailable on common software programs; strategies for measuring these properties vary; and it is often unclear how to define a clinically important change in patient status. Using data from a clinical trial of therapy for back pain, we demonstrate the calculation of several statistics for measuring reproducibility and responsiveness, and demonstrate relationships among them. Simple computational guides for several statistics are provided. We conclude that reproducibility should generally be quantified with the intraclass correlation coefficient rather than the more common Pearson r. Assessing reproducibility by retest at one-to-two week intervals (rather than a shorter interval) may result in more realistic estimates of the variability to be observed among control subjects in a longitudinal study. Instrument responsiveness should be quantified using indicators of effect size, a modified effect size statistic proposed by Guyatt, or the use of receiver operating characteristic (ROC) curves to describe how well various score changes can distinguish improved from unimproved patients.

KW - functional status

KW - Health status

KW - quality-of-life

KW - questionnaires

KW - responsiveness

UR - http://www.scopus.com/inward/record.url?scp=0025851102&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0025851102&partnerID=8YFLogxK

U2 - 10.1016/S0197-2456(05)80019-4

DO - 10.1016/S0197-2456(05)80019-4

M3 - Article

C2 - 1663851

AN - SCOPUS:0025851102

VL - 12

JO - Controlled Clinical Trials

JF - Controlled Clinical Trials

SN - 0197-2456

IS - 4 SUPPL.

ER -