A videotaped CIBIC for dementia patients: Validity and reliability in a simulated clinical trial

Joseph Quinn, M. Moore, D. F. Benson, C. M. Clark, R. Doody, W. Jagust, D. Knopman, Jeffrey Kaye

Research output: Contribution to journalArticle

37 Citations (Scopus)

Abstract

Background: The global impression of a clinician is an Food and Drug Administration-mandated primary outcome measure for clinical trials in dementia. Reliability and validity of these measures are not well established. Methods: A videotaped version of the Clinician's Interview Based Impression of Change (CIBIC) was evaluated. Raters were informed that the videotaped interviews were taken at baseline and 6 to 12 months later, when in fact half of the interviews were shown in reverse order. Ratings on "true order" interviews were compared with ratings on "reverse order" interviews. In addition, ratings by neurologists experienced in dementia were compared with those of less experienced raters. Results: Inter-rater reliability of the neurologists was poor when measured by absolute agreement on a 7-point scale (kappa = 0.18). With a less stringent 3-point scale (better, worse, or unchanged), inter-rater reliability was significantly better for the true order videos (kappa = 0.51) than for the reversed order videos (kappa = 0.12). Validity also was reduced in the reverse order group: neurologists rated 90% of subjects correctly in the "true order" group and 63% correctly in the "reversed order" group. The inter-rater reliability of the neurologists was greater than the less experienced raters, but the validity of the neurologists' ratings was only marginally better. Conclusions: The reliability and validity of the videotape CIBIC are reasonable when patients follow the expected course of gradual decline, but are poor when patients appear to improve. These findings suggest that global assessments should be modified as outcome measures in clinical trials with patients with dementia.

Original languageEnglish (US)
Pages (from-to)433-437
Number of pages5
JournalNeurology
Volume58
Issue number3
StatePublished - Feb 12 2002
Externally publishedYes

Fingerprint

Reproducibility of Results
Dementia
Clinical Trials
Interviews
Outcome Assessment (Health Care)
Videotape Recording
United States Food and Drug Administration
Neurologists

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Quinn, J., Moore, M., Benson, D. F., Clark, C. M., Doody, R., Jagust, W., ... Kaye, J. (2002). A videotaped CIBIC for dementia patients: Validity and reliability in a simulated clinical trial. Neurology, 58(3), 433-437.

A videotaped CIBIC for dementia patients : Validity and reliability in a simulated clinical trial. / Quinn, Joseph; Moore, M.; Benson, D. F.; Clark, C. M.; Doody, R.; Jagust, W.; Knopman, D.; Kaye, Jeffrey.

In: Neurology, Vol. 58, No. 3, 12.02.2002, p. 433-437.

Research output: Contribution to journalArticle

Quinn, J, Moore, M, Benson, DF, Clark, CM, Doody, R, Jagust, W, Knopman, D & Kaye, J 2002, 'A videotaped CIBIC for dementia patients: Validity and reliability in a simulated clinical trial', Neurology, vol. 58, no. 3, pp. 433-437.
Quinn J, Moore M, Benson DF, Clark CM, Doody R, Jagust W et al. A videotaped CIBIC for dementia patients: Validity and reliability in a simulated clinical trial. Neurology. 2002 Feb 12;58(3):433-437.
Quinn, Joseph ; Moore, M. ; Benson, D. F. ; Clark, C. M. ; Doody, R. ; Jagust, W. ; Knopman, D. ; Kaye, Jeffrey. / A videotaped CIBIC for dementia patients : Validity and reliability in a simulated clinical trial. In: Neurology. 2002 ; Vol. 58, No. 3. pp. 433-437.
@article{dfe576839af54cc7bf9a97c3be2b597f,
title = "A videotaped CIBIC for dementia patients: Validity and reliability in a simulated clinical trial",
abstract = "Background: The global impression of a clinician is an Food and Drug Administration-mandated primary outcome measure for clinical trials in dementia. Reliability and validity of these measures are not well established. Methods: A videotaped version of the Clinician's Interview Based Impression of Change (CIBIC) was evaluated. Raters were informed that the videotaped interviews were taken at baseline and 6 to 12 months later, when in fact half of the interviews were shown in reverse order. Ratings on {"}true order{"} interviews were compared with ratings on {"}reverse order{"} interviews. In addition, ratings by neurologists experienced in dementia were compared with those of less experienced raters. Results: Inter-rater reliability of the neurologists was poor when measured by absolute agreement on a 7-point scale (kappa = 0.18). With a less stringent 3-point scale (better, worse, or unchanged), inter-rater reliability was significantly better for the true order videos (kappa = 0.51) than for the reversed order videos (kappa = 0.12). Validity also was reduced in the reverse order group: neurologists rated 90{\%} of subjects correctly in the {"}true order{"} group and 63{\%} correctly in the {"}reversed order{"} group. The inter-rater reliability of the neurologists was greater than the less experienced raters, but the validity of the neurologists' ratings was only marginally better. Conclusions: The reliability and validity of the videotape CIBIC are reasonable when patients follow the expected course of gradual decline, but are poor when patients appear to improve. These findings suggest that global assessments should be modified as outcome measures in clinical trials with patients with dementia.",
author = "Joseph Quinn and M. Moore and Benson, {D. F.} and Clark, {C. M.} and R. Doody and W. Jagust and D. Knopman and Jeffrey Kaye",
year = "2002",
month = "2",
day = "12",
language = "English (US)",
volume = "58",
pages = "433--437",
journal = "Neurology",
issn = "0028-3878",
publisher = "Lippincott Williams and Wilkins",
number = "3",

}

TY - JOUR

T1 - A videotaped CIBIC for dementia patients

T2 - Validity and reliability in a simulated clinical trial

AU - Quinn, Joseph

AU - Moore, M.

AU - Benson, D. F.

AU - Clark, C. M.

AU - Doody, R.

AU - Jagust, W.

AU - Knopman, D.

AU - Kaye, Jeffrey

PY - 2002/2/12

Y1 - 2002/2/12

N2 - Background: The global impression of a clinician is an Food and Drug Administration-mandated primary outcome measure for clinical trials in dementia. Reliability and validity of these measures are not well established. Methods: A videotaped version of the Clinician's Interview Based Impression of Change (CIBIC) was evaluated. Raters were informed that the videotaped interviews were taken at baseline and 6 to 12 months later, when in fact half of the interviews were shown in reverse order. Ratings on "true order" interviews were compared with ratings on "reverse order" interviews. In addition, ratings by neurologists experienced in dementia were compared with those of less experienced raters. Results: Inter-rater reliability of the neurologists was poor when measured by absolute agreement on a 7-point scale (kappa = 0.18). With a less stringent 3-point scale (better, worse, or unchanged), inter-rater reliability was significantly better for the true order videos (kappa = 0.51) than for the reversed order videos (kappa = 0.12). Validity also was reduced in the reverse order group: neurologists rated 90% of subjects correctly in the "true order" group and 63% correctly in the "reversed order" group. The inter-rater reliability of the neurologists was greater than the less experienced raters, but the validity of the neurologists' ratings was only marginally better. Conclusions: The reliability and validity of the videotape CIBIC are reasonable when patients follow the expected course of gradual decline, but are poor when patients appear to improve. These findings suggest that global assessments should be modified as outcome measures in clinical trials with patients with dementia.

AB - Background: The global impression of a clinician is an Food and Drug Administration-mandated primary outcome measure for clinical trials in dementia. Reliability and validity of these measures are not well established. Methods: A videotaped version of the Clinician's Interview Based Impression of Change (CIBIC) was evaluated. Raters were informed that the videotaped interviews were taken at baseline and 6 to 12 months later, when in fact half of the interviews were shown in reverse order. Ratings on "true order" interviews were compared with ratings on "reverse order" interviews. In addition, ratings by neurologists experienced in dementia were compared with those of less experienced raters. Results: Inter-rater reliability of the neurologists was poor when measured by absolute agreement on a 7-point scale (kappa = 0.18). With a less stringent 3-point scale (better, worse, or unchanged), inter-rater reliability was significantly better for the true order videos (kappa = 0.51) than for the reversed order videos (kappa = 0.12). Validity also was reduced in the reverse order group: neurologists rated 90% of subjects correctly in the "true order" group and 63% correctly in the "reversed order" group. The inter-rater reliability of the neurologists was greater than the less experienced raters, but the validity of the neurologists' ratings was only marginally better. Conclusions: The reliability and validity of the videotape CIBIC are reasonable when patients follow the expected course of gradual decline, but are poor when patients appear to improve. These findings suggest that global assessments should be modified as outcome measures in clinical trials with patients with dementia.

UR - http://www.scopus.com/inward/record.url?scp=0037065762&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0037065762&partnerID=8YFLogxK

M3 - Article

C2 - 11839844

AN - SCOPUS:0037065762

VL - 58

SP - 433

EP - 437

JO - Neurology

JF - Neurology

SN - 0028-3878

IS - 3

ER -