A comparative analysis of pulmonary and critical care medicine guideline development methodologies

Noah C. Schoenberg, Alan F. Barker, John Bernardo, Robin R. Deterding, Jerrold J. Ellner, Dean R. Hess, Neil R. MacIntyre, Fernando J. Martinez, Kevin C. Wilson

Research output: Research - peer-reviewArticle

  • 1 Citations

Abstract

Rationale: The Institute of Medicine (IOM) standards for guideline development have had unintended negative consequences. A more efficient approach is desirable. Objectives: To determine whether a modified Delphi process early during guideline development discriminates recommendations that should be informed by a systematic review from those that can be based upon expert consensus. Methods: The same questions addressed by IOM-compliant pulmonary or critical care guidelines were addressed by expert panels using a modified Delphi process, termed the Convergence of Opinion on Recommendations and Evidence (CORE) process. The resulting recommendations were compared. Concordance of the course of action, strength of recommendation, and quality of evidence, as well as the duration of recommendation development, were measured. Measurements and Main Results: When 50% agreement was required to make a recommendation, all questions yielded recommendations, and the recommended courses of action were 89.6% concordant. When 70% agreement was required, 17.9% of questions did not yield recommendations, but for those that did, the recommended courses of action were 98.2% concordant. The time to completion was shorter for the CORE process (median, 19.3 vs. 1,309.0 d; P = 0.0002). Conclusions: We propose the CORE process as an early step in guideline creation. Questions for which 70% agreement on a recommendation cannot be achieved should go through an IOMcompliant process; however, questions for which 70% agreement on a recommendation can be achieved can be accepted, avoiding a lengthy systematic review.

LanguageEnglish (US)
Pages621-627
Number of pages7
JournalAmerican Journal of Respiratory and Critical Care Medicine
Volume196
Issue number5
DOIs
StatePublished - Sep 1 2017

Fingerprint

Critical Care
Medicine
Guidelines
Lung
National Academies of Science, Engineering, and Medicine (U.S.) Health and Medicine Division

Keywords

  • Assessment
  • Clinical practice guidelines
  • Development
  • Evaluation approach
  • Grading of Recommendations
  • Institute of Medicine standards for trustworthy guidelines
  • Methodology

ASJC Scopus subject areas

  • Pulmonary and Respiratory Medicine
  • Critical Care and Intensive Care Medicine

Cite this

A comparative analysis of pulmonary and critical care medicine guideline development methodologies. / Schoenberg, Noah C.; Barker, Alan F.; Bernardo, John; Deterding, Robin R.; Ellner, Jerrold J.; Hess, Dean R.; MacIntyre, Neil R.; Martinez, Fernando J.; Wilson, Kevin C.

In: American Journal of Respiratory and Critical Care Medicine, Vol. 196, No. 5, 01.09.2017, p. 621-627.

Research output: Research - peer-reviewArticle

Schoenberg, NC, Barker, AF, Bernardo, J, Deterding, RR, Ellner, JJ, Hess, DR, MacIntyre, NR, Martinez, FJ & Wilson, KC 2017, 'A comparative analysis of pulmonary and critical care medicine guideline development methodologies' American Journal of Respiratory and Critical Care Medicine, vol 196, no. 5, pp. 621-627. DOI: 10.1164/rccm.201705-0926OC
Schoenberg, Noah C. ; Barker, Alan F. ; Bernardo, John ; Deterding, Robin R. ; Ellner, Jerrold J. ; Hess, Dean R. ; MacIntyre, Neil R. ; Martinez, Fernando J. ; Wilson, Kevin C./ A comparative analysis of pulmonary and critical care medicine guideline development methodologies. In: American Journal of Respiratory and Critical Care Medicine. 2017 ; Vol. 196, No. 5. pp. 621-627
@article{46404d94b0874642bcb76ec654110fd0,
title = "A comparative analysis of pulmonary and critical care medicine guideline development methodologies",
abstract = "Rationale: The Institute of Medicine (IOM) standards for guideline development have had unintended negative consequences. A more efficient approach is desirable. Objectives: To determine whether a modified Delphi process early during guideline development discriminates recommendations that should be informed by a systematic review from those that can be based upon expert consensus. Methods: The same questions addressed by IOM-compliant pulmonary or critical care guidelines were addressed by expert panels using a modified Delphi process, termed the Convergence of Opinion on Recommendations and Evidence (CORE) process. The resulting recommendations were compared. Concordance of the course of action, strength of recommendation, and quality of evidence, as well as the duration of recommendation development, were measured. Measurements and Main Results: When 50% agreement was required to make a recommendation, all questions yielded recommendations, and the recommended courses of action were 89.6% concordant. When 70% agreement was required, 17.9% of questions did not yield recommendations, but for those that did, the recommended courses of action were 98.2% concordant. The time to completion was shorter for the CORE process (median, 19.3 vs. 1,309.0 d; P = 0.0002). Conclusions: We propose the CORE process as an early step in guideline creation. Questions for which 70% agreement on a recommendation cannot be achieved should go through an IOMcompliant process; however, questions for which 70% agreement on a recommendation can be achieved can be accepted, avoiding a lengthy systematic review.",
keywords = "Assessment, Clinical practice guidelines, Development, Evaluation approach, Grading of Recommendations, Institute of Medicine standards for trustworthy guidelines, Methodology",
author = "Schoenberg, {Noah C.} and Barker, {Alan F.} and John Bernardo and Deterding, {Robin R.} and Ellner, {Jerrold J.} and Hess, {Dean R.} and MacIntyre, {Neil R.} and Martinez, {Fernando J.} and Wilson, {Kevin C.}",
year = "2017",
month = "9",
doi = "10.1164/rccm.201705-0926OC",
volume = "196",
pages = "621--627",
journal = "American Journal of Respiratory and Critical Care Medicine",
issn = "1073-449X",
publisher = "American Thoracic Society",
number = "5",

}

TY - JOUR

T1 - A comparative analysis of pulmonary and critical care medicine guideline development methodologies

AU - Schoenberg,Noah C.

AU - Barker,Alan F.

AU - Bernardo,John

AU - Deterding,Robin R.

AU - Ellner,Jerrold J.

AU - Hess,Dean R.

AU - MacIntyre,Neil R.

AU - Martinez,Fernando J.

AU - Wilson,Kevin C.

PY - 2017/9/1

Y1 - 2017/9/1

N2 - Rationale: The Institute of Medicine (IOM) standards for guideline development have had unintended negative consequences. A more efficient approach is desirable. Objectives: To determine whether a modified Delphi process early during guideline development discriminates recommendations that should be informed by a systematic review from those that can be based upon expert consensus. Methods: The same questions addressed by IOM-compliant pulmonary or critical care guidelines were addressed by expert panels using a modified Delphi process, termed the Convergence of Opinion on Recommendations and Evidence (CORE) process. The resulting recommendations were compared. Concordance of the course of action, strength of recommendation, and quality of evidence, as well as the duration of recommendation development, were measured. Measurements and Main Results: When 50% agreement was required to make a recommendation, all questions yielded recommendations, and the recommended courses of action were 89.6% concordant. When 70% agreement was required, 17.9% of questions did not yield recommendations, but for those that did, the recommended courses of action were 98.2% concordant. The time to completion was shorter for the CORE process (median, 19.3 vs. 1,309.0 d; P = 0.0002). Conclusions: We propose the CORE process as an early step in guideline creation. Questions for which 70% agreement on a recommendation cannot be achieved should go through an IOMcompliant process; however, questions for which 70% agreement on a recommendation can be achieved can be accepted, avoiding a lengthy systematic review.

AB - Rationale: The Institute of Medicine (IOM) standards for guideline development have had unintended negative consequences. A more efficient approach is desirable. Objectives: To determine whether a modified Delphi process early during guideline development discriminates recommendations that should be informed by a systematic review from those that can be based upon expert consensus. Methods: The same questions addressed by IOM-compliant pulmonary or critical care guidelines were addressed by expert panels using a modified Delphi process, termed the Convergence of Opinion on Recommendations and Evidence (CORE) process. The resulting recommendations were compared. Concordance of the course of action, strength of recommendation, and quality of evidence, as well as the duration of recommendation development, were measured. Measurements and Main Results: When 50% agreement was required to make a recommendation, all questions yielded recommendations, and the recommended courses of action were 89.6% concordant. When 70% agreement was required, 17.9% of questions did not yield recommendations, but for those that did, the recommended courses of action were 98.2% concordant. The time to completion was shorter for the CORE process (median, 19.3 vs. 1,309.0 d; P = 0.0002). Conclusions: We propose the CORE process as an early step in guideline creation. Questions for which 70% agreement on a recommendation cannot be achieved should go through an IOMcompliant process; however, questions for which 70% agreement on a recommendation can be achieved can be accepted, avoiding a lengthy systematic review.

KW - Assessment

KW - Clinical practice guidelines

KW - Development

KW - Evaluation approach

KW - Grading of Recommendations

KW - Institute of Medicine standards for trustworthy guidelines

KW - Methodology

UR - http://www.scopus.com/inward/record.url?scp=85029507095&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029507095&partnerID=8YFLogxK

U2 - 10.1164/rccm.201705-0926OC

DO - 10.1164/rccm.201705-0926OC

M3 - Article

VL - 196

SP - 621

EP - 627

JO - American Journal of Respiratory and Critical Care Medicine

T2 - American Journal of Respiratory and Critical Care Medicine

JF - American Journal of Respiratory and Critical Care Medicine

SN - 1073-449X

IS - 5

ER -